2026-02-23T10:17:33.650 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-23T10:17:33.665 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-23T10:17:33.748 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial/66074 branch: main description: crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/cls} email: null first_in_suite: false flavor: debug job_id: '66074' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug objclass: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) sha1: e371433fcd1390a0b82cfb18effc422c1f5d8c1d ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: debug sha1: e371433fcd1390a0b82cfb18effc422c1f5d8c1d workunit: branch: wip-shraddhaag-enable-cephadm-tests env: CRIMSON_COMPAT: '1' sha1: e371433fcd1390a0b82cfb18effc422c1f5d8c1d owner: scheduled_shraddhaag@soko04.front.sepia.ceph.com priority: 49 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 66 sha1: e371433fcd1390a0b82cfb18effc422c1f5d8c1d sleep_before_teardown: 0 suite: crimson-rados suite_branch: wip-shraddhaag-enable-cephadm-tests suite_path: /home/teuthworker/src/github.com_shraddhaag_ceph_e371433fcd1390a0b82cfb18effc422c1f5d8c1d/qa suite_relpath: qa suite_repo: https://github.com/shraddhaag/ceph.git suite_sha1: e371433fcd1390a0b82cfb18effc422c1f5d8c1d targets: trial038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtfCwuckGkN6i1n7kv7PxQkFv2itGL13cIm1t1CIrPT/6vdzmqLsPE67bnoxeG/1ALqD4uTtQguyHDSV9NMNrY= trial153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJogg+T3xVpsEzYIoOYY8NxCM23gi/MYPU21fpKfPtAQACeva3HTLKbRtdu0NMj5PBkCk1PA9X1dVlfDopSaMRk= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 - workunit: clients: client.0: - cls/test_cls_cmpomap.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-23_09:58:48 tube: trial user: shraddhaag verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653 2026-02-23T10:17:33.749 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_shraddhaag_ceph_e371433fcd1390a0b82cfb18effc422c1f5d8c1d/qa; will attempt to use it 2026-02-23T10:17:33.749 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_shraddhaag_ceph_e371433fcd1390a0b82cfb18effc422c1f5d8c1d/qa/tasks 2026-02-23T10:17:33.749 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-23T10:17:33.750 INFO:teuthology.task.internal:Checking packages... 2026-02-23T10:17:33.751 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d' 2026-02-23T10:17:33.751 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-23T10:17:33.751 INFO:teuthology.packaging:ref: None 2026-02-23T10:17:33.751 INFO:teuthology.packaging:tag: None 2026-02-23T10:17:33.751 INFO:teuthology.packaging:branch: main 2026-02-23T10:17:33.751 INFO:teuthology.packaging:sha1: e371433fcd1390a0b82cfb18effc422c1f5d8c1d 2026-02-23T10:17:33.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=main 2026-02-23T10:17:33.886 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5253.g5ccdd6ad 2026-02-23T10:17:33.887 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-23T10:17:33.899 INFO:teuthology.task.internal:no buildpackages task found 2026-02-23T10:17:33.899 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-23T10:17:33.913 INFO:teuthology.task.internal:Saving configuration 2026-02-23T10:17:33.926 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-23T10:17:33.938 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-23T10:17:34.021 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial038.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial/66074', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-23 10:14:03.799749', 'locked_by': 'scheduled_shraddhaag@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtfCwuckGkN6i1n7kv7PxQkFv2itGL13cIm1t1CIrPT/6vdzmqLsPE67bnoxeG/1ALqD4uTtQguyHDSV9NMNrY='} 2026-02-23T10:17:34.078 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial153.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial/66074', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-23 10:14:03.799134', 'locked_by': 'scheduled_shraddhaag@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJogg+T3xVpsEzYIoOYY8NxCM23gi/MYPU21fpKfPtAQACeva3HTLKbRtdu0NMj5PBkCk1PA9X1dVlfDopSaMRk='} 2026-02-23T10:17:34.078 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-23T10:17:34.092 INFO:teuthology.task.internal:roles: ubuntu@trial038.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2026-02-23T10:17:34.092 INFO:teuthology.task.internal:roles: ubuntu@trial153.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2026-02-23T10:17:34.092 INFO:teuthology.run_tasks:Running task console_log... 2026-02-23T10:17:34.229 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7c55dc41c540>, signals=[15]) 2026-02-23T10:17:34.229 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-23T10:17:34.239 INFO:teuthology.task.internal:Opening connections... 2026-02-23T10:17:34.239 DEBUG:teuthology.task.internal:connecting to ubuntu@trial038.front.sepia.ceph.com 2026-02-23T10:17:34.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-23T10:17:34.300 DEBUG:teuthology.task.internal:connecting to ubuntu@trial153.front.sepia.ceph.com 2026-02-23T10:17:34.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-23T10:17:34.361 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-23T10:17:34.371 DEBUG:teuthology.orchestra.run.trial038:> uname -m 2026-02-23T10:17:34.387 INFO:teuthology.orchestra.run.trial038.stdout:x86_64 2026-02-23T10:17:34.387 DEBUG:teuthology.orchestra.run.trial038:> cat /etc/os-release 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:NAME="CentOS Stream" 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:VERSION="9" 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:ID="centos" 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:ID_LIKE="rhel fedora" 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:VERSION_ID="9" 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:PLATFORM_ID="platform:el9" 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:ANSI_COLOR="0;31" 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:LOGO="fedora-logo-icon" 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:HOME_URL="https://centos.org/" 2026-02-23T10:17:34.443 INFO:teuthology.orchestra.run.trial038.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-23T10:17:34.444 INFO:teuthology.orchestra.run.trial038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-23T10:17:34.444 INFO:teuthology.orchestra.run.trial038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-23T10:17:34.444 INFO:teuthology.lock.ops:Updating trial038.front.sepia.ceph.com on lock server 2026-02-23T10:17:34.512 DEBUG:teuthology.orchestra.run.trial153:> uname -m 2026-02-23T10:17:34.527 INFO:teuthology.orchestra.run.trial153.stdout:x86_64 2026-02-23T10:17:34.527 DEBUG:teuthology.orchestra.run.trial153:> cat /etc/os-release 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:NAME="CentOS Stream" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:VERSION="9" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:ID="centos" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:ID_LIKE="rhel fedora" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:VERSION_ID="9" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:PLATFORM_ID="platform:el9" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:ANSI_COLOR="0;31" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:LOGO="fedora-logo-icon" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:HOME_URL="https://centos.org/" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-23T10:17:34.583 INFO:teuthology.orchestra.run.trial153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-23T10:17:34.584 INFO:teuthology.lock.ops:Updating trial153.front.sepia.ceph.com on lock server 2026-02-23T10:17:34.671 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-23T10:17:34.682 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-23T10:17:34.694 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-23T10:17:34.694 DEBUG:teuthology.orchestra.run.trial038:> test '!' -e /home/ubuntu/cephtest 2026-02-23T10:17:34.696 DEBUG:teuthology.orchestra.run.trial153:> test '!' -e /home/ubuntu/cephtest 2026-02-23T10:17:34.711 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-23T10:17:34.722 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-23T10:17:34.722 DEBUG:teuthology.orchestra.run.trial038:> test -z $(ls -A /var/lib/ceph) 2026-02-23T10:17:34.752 DEBUG:teuthology.orchestra.run.trial153:> test -z $(ls -A /var/lib/ceph) 2026-02-23T10:17:34.769 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-23T10:17:34.934 INFO:teuthology.run_tasks:Running task kernel... 2026-02-23T10:17:34.953 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-23T10:17:34.953 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-23T10:17:34.953 DEBUG:teuthology.orchestra.run.trial038:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-23T10:17:34.953 DEBUG:teuthology.orchestra.run.trial153:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-23T10:17:34.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T10:17:34.968 DEBUG:teuthology.orchestra.run.trial153:> uname -r 2026-02-23T10:17:34.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T10:17:34.968 DEBUG:teuthology.orchestra.run.trial038:> uname -r 2026-02-23T10:17:35.022 INFO:teuthology.orchestra.run.trial038.stdout:5.14.0-681.el9.x86_64 2026-02-23T10:17:35.022 INFO:teuthology.task.kernel:Running kernel on trial038: 5.14.0-681.el9.x86_64 2026-02-23T10:17:35.022 DEBUG:teuthology.orchestra.run.trial038:> sudo yum install -y kernel 2026-02-23T10:17:35.023 INFO:teuthology.orchestra.run.trial153.stdout:5.14.0-681.el9.x86_64 2026-02-23T10:17:35.024 INFO:teuthology.task.kernel:Running kernel on trial153: 5.14.0-681.el9.x86_64 2026-02-23T10:17:35.024 DEBUG:teuthology.orchestra.run.trial153:> sudo yum install -y kernel 2026-02-23T10:17:36.203 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-23T10:17:36.837 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.9 MB 00:01 2026-02-23T10:17:39.948 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 27 MB 00:03 2026-02-23T10:17:39.980 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-23T10:17:43.218 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 7.8 MB 00:00 2026-02-23T10:17:43.488 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - CRB 6.8 MB/s | 7.8 MB 00:01 2026-02-23T10:17:44.044 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - Extras packages 195 kB/s | 20 kB 00:00 2026-02-23T10:17:44.452 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 20 kB 00:00 2026-02-23T10:17:45.228 INFO:teuthology.orchestra.run.trial153.stdout:Extra Packages for Enterprise Linux 18 MB/s | 20 MB 00:01 2026-02-23T10:17:45.649 INFO:teuthology.orchestra.run.trial038.stdout:Extra Packages for Enterprise Linux 18 MB/s | 20 MB 00:01 2026-02-23T10:17:48.411 INFO:teuthology.orchestra.run.trial153.stdout:lab-extras 2.3 MB/s | 50 kB 00:00 2026-02-23T10:17:48.789 INFO:teuthology.orchestra.run.trial038.stdout:lab-extras 2.0 MB/s | 50 kB 00:00 2026-02-23T10:17:49.419 INFO:teuthology.orchestra.run.trial153.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-23T10:17:49.419 INFO:teuthology.orchestra.run.trial153.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-23T10:17:49.438 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:17:49.438 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:17:49.438 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:17:49.462 DEBUG:teuthology.orchestra.run.trial153:> echo no | sudo yum reinstall kernel || true 2026-02-23T10:17:49.793 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:01 ago on Mon 23 Feb 2026 10:17:48 AM UTC. 2026-02-23T10:17:49.806 INFO:teuthology.orchestra.run.trial038.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-23T10:17:49.807 INFO:teuthology.orchestra.run.trial038.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-23T10:17:49.826 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:17:49.826 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:17:49.826 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:17:49.850 DEBUG:teuthology.orchestra.run.trial038:> echo no | sudo yum reinstall kernel || true 2026-02-23T10:17:49.872 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:17:49.873 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:17:49.873 INFO:teuthology.orchestra.run.trial153.stdout: Package Architecture Version Repository Size 2026-02-23T10:17:49.873 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:17:49.873 INFO:teuthology.orchestra.run.trial153.stdout:Reinstalling: 2026-02-23T10:17:49.873 INFO:teuthology.orchestra.run.trial153.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-23T10:17:49.873 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:17:49.874 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:17:49.874 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:17:49.874 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:17:49.874 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 803 k 2026-02-23T10:17:49.874 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 0 2026-02-23T10:17:49.874 INFO:teuthology.orchestra.run.trial153.stderr:Operation aborted. 2026-02-23T10:17:49.893 INFO:teuthology.orchestra.run.trial153.stdout:Is this ok [y/N]: 2026-02-23T10:17:49.893 DEBUG:teuthology.orchestra.run.trial153:> sudo yum reinstall -y kernel || true 2026-02-23T10:17:50.150 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:02 ago on Mon 23 Feb 2026 10:17:48 AM UTC. 2026-02-23T10:17:50.190 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:02 ago on Mon 23 Feb 2026 10:17:48 AM UTC. 2026-02-23T10:17:50.230 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:17:50.231 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:17:50.231 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-23T10:17:50.231 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:17:50.231 INFO:teuthology.orchestra.run.trial038.stdout:Reinstalling: 2026-02-23T10:17:50.231 INFO:teuthology.orchestra.run.trial038.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-23T10:17:50.231 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:17:50.231 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:17:50.231 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:17:50.231 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:17:50.232 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 803 k 2026-02-23T10:17:50.232 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 0 2026-02-23T10:17:50.232 INFO:teuthology.orchestra.run.trial038.stderr:Operation aborted. 2026-02-23T10:17:50.251 INFO:teuthology.orchestra.run.trial038.stdout:Is this ok [y/N]: 2026-02-23T10:17:50.251 DEBUG:teuthology.orchestra.run.trial038:> sudo yum reinstall -y kernel || true 2026-02-23T10:17:50.270 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:17:50.271 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:17:50.271 INFO:teuthology.orchestra.run.trial153.stdout: Package Architecture Version Repository Size 2026-02-23T10:17:50.271 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:17:50.272 INFO:teuthology.orchestra.run.trial153.stdout:Reinstalling: 2026-02-23T10:17:50.272 INFO:teuthology.orchestra.run.trial153.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-23T10:17:50.272 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:17:50.272 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:17:50.272 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:17:50.272 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:17:50.272 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 803 k 2026-02-23T10:17:50.272 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 0 2026-02-23T10:17:50.272 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:17:50.548 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:02 ago on Mon 23 Feb 2026 10:17:48 AM UTC. 2026-02-23T10:17:50.606 INFO:teuthology.orchestra.run.trial153.stdout:kernel-5.14.0-681.el9.x86_64.rpm 29 MB/s | 803 kB 00:00 2026-02-23T10:17:50.606 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:17:50.606 INFO:teuthology.orchestra.run.trial153.stdout:Total 2.3 MB/s | 803 kB 00:00 2026-02-23T10:17:50.615 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:17:50.627 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:17:50.628 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:17:50.628 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-23T10:17:50.628 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:17:50.628 INFO:teuthology.orchestra.run.trial038.stdout:Reinstalling: 2026-02-23T10:17:50.629 INFO:teuthology.orchestra.run.trial038.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-23T10:17:50.629 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:17:50.629 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:17:50.629 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:17:50.629 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:17:50.629 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 803 k 2026-02-23T10:17:50.629 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 0 2026-02-23T10:17:50.629 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:17:50.649 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:17:50.649 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:17:50.653 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:17:50.654 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:17:50.677 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:17:50.682 INFO:teuthology.orchestra.run.trial153.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-23T10:17:50.682 INFO:teuthology.orchestra.run.trial153.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-23T10:17:50.794 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-23T10:17:50.795 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-23T10:17:50.846 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-23T10:17:50.846 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:17:50.846 INFO:teuthology.orchestra.run.trial153.stdout:Reinstalled: 2026-02-23T10:17:50.846 INFO:teuthology.orchestra.run.trial153.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-23T10:17:50.846 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:17:50.846 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:17:50.874 DEBUG:teuthology.orchestra.run.trial153:> rpm -q kernel | sort -rV | head -n 1 2026-02-23T10:17:50.898 INFO:teuthology.orchestra.run.trial153.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-23T10:17:50.898 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-23T10:17:50.898 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-23T10:17:50.899 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2026-02-23T10:17:50.899 DEBUG:teuthology.orchestra.run.trial153:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-23T10:17:50.961 INFO:teuthology.orchestra.run.trial153.stdout:ttyS1 2026-02-23T10:17:50.982 DEBUG:teuthology.parallel:result is None 2026-02-23T10:17:53.032 INFO:teuthology.orchestra.run.trial038.stdout:kernel-5.14.0-681.el9.x86_64.rpm 343 kB/s | 803 kB 00:02 2026-02-23T10:17:53.033 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:17:53.033 INFO:teuthology.orchestra.run.trial038.stdout:Total 334 kB/s | 803 kB 00:02 2026-02-23T10:17:53.043 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:17:53.076 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:17:53.076 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:17:53.081 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:17:53.081 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:17:53.104 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:17:53.109 INFO:teuthology.orchestra.run.trial038.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-23T10:17:53.109 INFO:teuthology.orchestra.run.trial038.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-23T10:17:53.217 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-23T10:17:53.217 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-23T10:17:53.275 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-23T10:17:53.275 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:17:53.275 INFO:teuthology.orchestra.run.trial038.stdout:Reinstalled: 2026-02-23T10:17:53.275 INFO:teuthology.orchestra.run.trial038.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-23T10:17:53.275 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:17:53.276 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:17:53.302 DEBUG:teuthology.orchestra.run.trial038:> rpm -q kernel | sort -rV | head -n 1 2026-02-23T10:17:53.326 INFO:teuthology.orchestra.run.trial038.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-23T10:17:53.328 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-23T10:17:53.328 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-23T10:17:53.328 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-23T10:17:53.328 DEBUG:teuthology.orchestra.run.trial038:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-23T10:17:53.392 INFO:teuthology.orchestra.run.trial038.stdout:ttyS1 2026-02-23T10:17:53.414 DEBUG:teuthology.parallel:result is None 2026-02-23T10:17:53.414 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-23T10:17:53.425 INFO:teuthology.task.internal:Creating test directory... 2026-02-23T10:17:53.426 DEBUG:teuthology.orchestra.run.trial038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-23T10:17:53.456 DEBUG:teuthology.orchestra.run.trial153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-23T10:17:53.473 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-23T10:17:53.485 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-23T10:17:53.501 INFO:teuthology.task.internal:Creating archive directory... 2026-02-23T10:17:53.502 DEBUG:teuthology.orchestra.run.trial038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-23T10:17:53.512 DEBUG:teuthology.orchestra.run.trial153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-23T10:17:53.535 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-23T10:17:53.548 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-23T10:17:53.548 DEBUG:teuthology.orchestra.run.trial038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-23T10:17:53.569 DEBUG:teuthology.orchestra.run.trial153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-23T10:17:53.591 INFO:teuthology.orchestra.run.trial038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T10:17:53.594 INFO:teuthology.orchestra.run.trial153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T10:17:53.600 INFO:teuthology.orchestra.run.trial038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T10:17:53.602 INFO:teuthology.orchestra.run.trial153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T10:17:53.604 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-23T10:17:53.614 INFO:teuthology.task.internal:Configuring sudo... 2026-02-23T10:17:53.615 DEBUG:teuthology.orchestra.run.trial038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-23T10:17:53.644 DEBUG:teuthology.orchestra.run.trial153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-23T10:17:53.667 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-23T10:17:53.682 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-23T10:17:53.682 DEBUG:teuthology.orchestra.run.trial038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-23T10:17:53.708 DEBUG:teuthology.orchestra.run.trial153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-23T10:17:53.724 DEBUG:teuthology.orchestra.run.trial038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-23T10:17:53.884 DEBUG:teuthology.orchestra.run.trial038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-23T10:17:53.907 DEBUG:teuthology.orchestra.run.trial038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-23T10:17:54.059 DEBUG:teuthology.orchestra.run.trial038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-23T10:17:54.082 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:17:54.082 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-23T10:17:54.145 DEBUG:teuthology.orchestra.run.trial153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-23T10:17:54.235 DEBUG:teuthology.orchestra.run.trial153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-23T10:17:54.259 DEBUG:teuthology.orchestra.run.trial153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-23T10:17:54.386 DEBUG:teuthology.orchestra.run.trial153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-23T10:17:54.409 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:17:54.409 DEBUG:teuthology.orchestra.run.trial153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-23T10:17:54.472 DEBUG:teuthology.orchestra.run.trial038:> sudo service rsyslog restart 2026-02-23T10:17:54.474 DEBUG:teuthology.orchestra.run.trial153:> sudo service rsyslog restart 2026-02-23T10:17:54.501 INFO:teuthology.orchestra.run.trial038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-23T10:17:54.539 INFO:teuthology.orchestra.run.trial153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-23T10:17:54.876 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-23T10:17:54.889 INFO:teuthology.task.internal:Starting timer... 2026-02-23T10:17:54.889 INFO:teuthology.run_tasks:Running task pcp... 2026-02-23T10:17:54.910 INFO:teuthology.run_tasks:Running task selinux... 2026-02-23T10:17:54.928 DEBUG:teuthology.orchestra.run.trial038:> sudo service auditd rotate 2026-02-23T10:17:54.957 INFO:teuthology.orchestra.run.trial038.stdout:Rotating logs: 2026-02-23T10:17:54.959 DEBUG:teuthology.orchestra.run.trial153:> sudo service auditd rotate 2026-02-23T10:17:54.988 INFO:teuthology.orchestra.run.trial153.stdout:Rotating logs: 2026-02-23T10:17:54.990 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-23T10:17:54.990 DEBUG:teuthology.orchestra.run.trial038:> /usr/sbin/getenforce 2026-02-23T10:17:55.015 INFO:teuthology.orchestra.run.trial038.stdout:Permissive 2026-02-23T10:17:55.016 DEBUG:teuthology.orchestra.run.trial153:> /usr/sbin/getenforce 2026-02-23T10:17:55.046 INFO:teuthology.orchestra.run.trial153.stdout:Permissive 2026-02-23T10:17:55.046 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial038.front.sepia.ceph.com': 'permissive', 'ubuntu@trial153.front.sepia.ceph.com': 'permissive'} 2026-02-23T10:17:55.047 DEBUG:teuthology.orchestra.run.trial038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-23T10:17:55.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T10:17:55.080 DEBUG:teuthology.orchestra.run.trial153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-23T10:17:55.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T10:17:55.111 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-23T10:17:55.111 DEBUG:teuthology.orchestra.run.trial038:> sudo /usr/sbin/setenforce permissive 2026-02-23T10:17:55.144 DEBUG:teuthology.orchestra.run.trial153:> sudo /usr/sbin/setenforce permissive 2026-02-23T10:17:55.174 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-23T10:17:55.187 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-23T10:17:55.195 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-23T10:17:55.195 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial038.front.sepia.ceph.com,trial153.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-23T10:20:39.260 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial038.front.sepia.ceph.com'), Remote(name='ubuntu@trial153.front.sepia.ceph.com')] 2026-02-23T10:20:39.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial038.front.sepia.ceph.com' 2026-02-23T10:20:39.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-23T10:20:39.371 DEBUG:teuthology.orchestra.run.trial038:> true 2026-02-23T10:20:39.447 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial038.front.sepia.ceph.com' 2026-02-23T10:20:39.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial153.front.sepia.ceph.com' 2026-02-23T10:20:39.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-23T10:20:39.540 DEBUG:teuthology.orchestra.run.trial153:> true 2026-02-23T10:20:39.616 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial153.front.sepia.ceph.com' 2026-02-23T10:20:39.616 INFO:teuthology.run_tasks:Running task clock... 2026-02-23T10:20:39.629 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-23T10:20:39.630 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-23T10:20:39.630 DEBUG:teuthology.orchestra.run.trial038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-23T10:20:39.632 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-23T10:20:39.633 DEBUG:teuthology.orchestra.run.trial153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-23T10:20:39.658 INFO:teuthology.orchestra.run.trial038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-23T10:20:39.671 INFO:teuthology.orchestra.run.trial038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-23T10:20:39.683 INFO:teuthology.orchestra.run.trial153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-23T10:20:39.692 INFO:teuthology.orchestra.run.trial038.stderr:sudo: ntpd: command not found 2026-02-23T10:20:39.694 INFO:teuthology.orchestra.run.trial153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-23T10:20:39.701 INFO:teuthology.orchestra.run.trial038.stdout:506 Cannot talk to daemon 2026-02-23T10:20:39.713 INFO:teuthology.orchestra.run.trial038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-23T10:20:39.713 INFO:teuthology.orchestra.run.trial153.stderr:sudo: ntpd: command not found 2026-02-23T10:20:39.723 INFO:teuthology.orchestra.run.trial153.stdout:506 Cannot talk to daemon 2026-02-23T10:20:39.724 INFO:teuthology.orchestra.run.trial038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-23T10:20:39.734 INFO:teuthology.orchestra.run.trial153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-23T10:20:39.745 INFO:teuthology.orchestra.run.trial153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-23T10:20:39.770 INFO:teuthology.orchestra.run.trial038.stderr:bash: line 1: ntpq: command not found 2026-02-23T10:20:39.788 INFO:teuthology.orchestra.run.trial153.stderr:bash: line 1: ntpq: command not found 2026-02-23T10:20:39.789 INFO:teuthology.orchestra.run.trial038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-23T10:20:39.789 INFO:teuthology.orchestra.run.trial038.stdout:=============================================================================== 2026-02-23T10:20:39.789 INFO:teuthology.orchestra.run.trial038.stdout:^? 144.202.66.214.vultruser> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-23T10:20:39.789 INFO:teuthology.orchestra.run.trial038.stdout:^? gwallaceNTPpool.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-23T10:20:39.789 INFO:teuthology.orchestra.run.trial038.stdout:^? ntp4.glypnod.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-23T10:20:39.789 INFO:teuthology.orchestra.run.trial038.stdout:^? 23-93-251-54.dedicated.s> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-23T10:20:39.791 INFO:teuthology.orchestra.run.trial153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-23T10:20:39.791 INFO:teuthology.orchestra.run.trial153.stdout:=============================================================================== 2026-02-23T10:20:39.791 INFO:teuthology.orchestra.run.trial153.stdout:^? ip216-250-115-174.pbiaas> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-23T10:20:39.791 INFO:teuthology.orchestra.run.trial153.stdout:^? 23.186.168.130 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-23T10:20:39.791 INFO:teuthology.orchestra.run.trial153.stdout:^? 234.201.68.199.in-addr.a> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-23T10:20:39.791 INFO:teuthology.orchestra.run.trial153.stdout:^? chi3.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-23T10:20:39.792 INFO:teuthology.run_tasks:Running task install... 2026-02-23T10:20:39.806 DEBUG:teuthology.task.install:project ceph 2026-02-23T10:20:39.806 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d'}} 2026-02-23T10:20:39.806 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d'} 2026-02-23T10:20:39.806 INFO:teuthology.task.install:Using flavor: debug 2026-02-23T10:20:39.815 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-23T10:20:39.816 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-23T10:20:39.816 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d', 'tag': None, 'wait_for_package': False} 2026-02-23T10:20:39.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e371433fcd1390a0b82cfb18effc422c1f5d8c1d 2026-02-23T10:20:39.819 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d', 'tag': None, 'wait_for_package': False} 2026-02-23T10:20:39.820 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e371433fcd1390a0b82cfb18effc422c1f5d8c1d 2026-02-23T10:20:40.007 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/ 2026-02-23T10:20:40.008 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5439.ge371433f 2026-02-23T10:20:40.019 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/ 2026-02-23T10:20:40.019 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5439.ge371433f 2026-02-23T10:20:40.338 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-23T10:20:40.338 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:20:40.338 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-23T10:20:40.364 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-23T10:20:40.365 DEBUG:teuthology.orchestra.run.trial038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-23T10:20:40.377 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-23T10:20:40.377 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:20:40.377 DEBUG:teuthology.orchestra.run.trial153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-23T10:20:40.404 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-23T10:20:40.404 DEBUG:teuthology.orchestra.run.trial153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-23T10:20:40.429 DEBUG:teuthology.orchestra.run.trial038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-23T10:20:40.469 DEBUG:teuthology.orchestra.run.trial153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-23T10:20:40.503 DEBUG:teuthology.orchestra.run.trial038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-23T10:20:40.541 DEBUG:teuthology.orchestra.run.trial153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-23T10:20:40.567 INFO:teuthology.orchestra.run.trial038.stdout:check_obsoletes = 1 2026-02-23T10:20:40.568 DEBUG:teuthology.orchestra.run.trial038:> sudo yum clean all 2026-02-23T10:20:40.606 INFO:teuthology.orchestra.run.trial153.stdout:check_obsoletes = 1 2026-02-23T10:20:40.607 DEBUG:teuthology.orchestra.run.trial153:> sudo yum clean all 2026-02-23T10:20:40.728 INFO:teuthology.orchestra.run.trial038.stdout:41 files removed 2026-02-23T10:20:40.742 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-radosgw 2026-02-23T10:20:40.766 INFO:teuthology.orchestra.run.trial153.stdout:41 files removed 2026-02-23T10:20:40.781 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-radosgw 2026-02-23T10:20:41.513 INFO:teuthology.orchestra.run.trial038.stdout:ceph packages for x86_64 146 kB/s | 92 kB 00:00 2026-02-23T10:20:41.565 INFO:teuthology.orchestra.run.trial153.stdout:ceph packages for x86_64 143 kB/s | 92 kB 00:00 2026-02-23T10:20:42.123 INFO:teuthology.orchestra.run.trial038.stdout:ceph noarch packages 41 kB/s | 24 kB 00:00 2026-02-23T10:20:42.177 INFO:teuthology.orchestra.run.trial153.stdout:ceph noarch packages 40 kB/s | 24 kB 00:00 2026-02-23T10:20:42.689 INFO:teuthology.orchestra.run.trial038.stdout:ceph source packages 3.8 kB/s | 2.1 kB 00:00 2026-02-23T10:20:42.746 INFO:teuthology.orchestra.run.trial153.stdout:ceph source packages 3.8 kB/s | 2.1 kB 00:00 2026-02-23T10:20:43.159 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.9 MB 00:00 2026-02-23T10:20:43.593 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-23T10:20:46.379 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 27 MB 00:02 2026-02-23T10:20:46.637 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-23T10:20:50.044 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 7.8 MB 00:01 2026-02-23T10:20:50.583 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 7.8 MB 00:01 2026-02-23T10:20:50.958 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - Extras packages 122 kB/s | 20 kB 00:00 2026-02-23T10:20:51.490 INFO:teuthology.orchestra.run.trial153.stdout:Extra Packages for Enterprise Linux 43 MB/s | 20 MB 00:00 2026-02-23T10:20:51.744 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 20 kB 00:00 2026-02-23T10:20:52.952 INFO:teuthology.orchestra.run.trial038.stdout:Extra Packages for Enterprise Linux 18 MB/s | 20 MB 00:01 2026-02-23T10:20:54.720 INFO:teuthology.orchestra.run.trial153.stdout:lab-extras 1.6 MB/s | 50 kB 00:00 2026-02-23T10:20:55.916 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:20:55.917 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:20:55.917 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repository Size 2026-02-23T10:20:55.917 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:20:55.917 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout: ceph-radosgw x86_64 2:20.3.0-5439.ge371433f.el9 ceph 11 M 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout:Upgrading: 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout: librados2 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 3.3 M 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout: librbd1 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 2.7 M 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout:Installing dependencies: 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout: ceph-base x86_64 2:20.3.0-5439.ge371433f.el9 ceph 5.1 M 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout: ceph-common x86_64 2:20.3.0-5439.ge371433f.el9 ceph 21 M 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout: ceph-selinux x86_64 2:20.3.0-5439.ge371433f.el9 ceph 25 k 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-23T10:20:55.918 INFO:teuthology.orchestra.run.trial153.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: libcephfs2 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 860 k 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: libradosstriper1 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 263 k 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: librgw2 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 5.6 M 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-23T10:20:55.919 INFO:teuthology.orchestra.run.trial153.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: python3-ceph-argparse x86_64 2:20.3.0-5439.ge371433f.el9 ceph 45 k 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: python3-ceph-common x86_64 2:20.3.0-5439.ge371433f.el9 ceph 186 k 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: python3-cephfs x86_64 2:20.3.0-5439.ge371433f.el9 ceph 204 k 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: python3-rados x86_64 2:20.3.0-5439.ge371433f.el9 ceph 324 k 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: python3-rbd x86_64 2:20.3.0-5439.ge371433f.el9 ceph 305 k 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: python3-rgw x86_64 2:20.3.0-5439.ge371433f.el9 ceph 100 k 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout:Installing weak dependencies: 2026-02-23T10:20:55.920 INFO:teuthology.orchestra.run.trial153.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-23T10:20:55.921 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:20:55.921 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:20:55.921 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:20:55.921 INFO:teuthology.orchestra.run.trial153.stdout:Install 32 Packages 2026-02-23T10:20:55.921 INFO:teuthology.orchestra.run.trial153.stdout:Upgrade 2 Packages 2026-02-23T10:20:55.921 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:20:55.922 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 60 M 2026-02-23T10:20:55.922 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:20:56.137 INFO:teuthology.orchestra.run.trial038.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-23T10:20:57.343 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:20:57.344 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:20:57.344 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-23T10:20:57.344 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:20:57.344 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:20:57.344 INFO:teuthology.orchestra.run.trial038.stdout: ceph-radosgw x86_64 2:20.3.0-5439.ge371433f.el9 ceph 11 M 2026-02-23T10:20:57.344 INFO:teuthology.orchestra.run.trial038.stdout:Upgrading: 2026-02-23T10:20:57.344 INFO:teuthology.orchestra.run.trial038.stdout: librados2 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 3.3 M 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout: librbd1 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 2.7 M 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout: ceph-base x86_64 2:20.3.0-5439.ge371433f.el9 ceph 5.1 M 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout: ceph-common x86_64 2:20.3.0-5439.ge371433f.el9 ceph 21 M 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout: ceph-selinux x86_64 2:20.3.0-5439.ge371433f.el9 ceph 25 k 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs2 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 860 k 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-23T10:20:57.345 INFO:teuthology.orchestra.run.trial038.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: libradosstriper1 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 263 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: librgw2 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 5.6 M 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-argparse x86_64 2:20.3.0-5439.ge371433f.el9 ceph 45 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-common x86_64 2:20.3.0-5439.ge371433f.el9 ceph 186 k 2026-02-23T10:20:57.346 INFO:teuthology.orchestra.run.trial038.stdout: python3-cephfs x86_64 2:20.3.0-5439.ge371433f.el9 ceph 204 k 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout: python3-rados x86_64 2:20.3.0-5439.ge371433f.el9 ceph 324 k 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout: python3-rbd x86_64 2:20.3.0-5439.ge371433f.el9 ceph 305 k 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout: python3-rgw x86_64 2:20.3.0-5439.ge371433f.el9 ceph 100 k 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout:Installing weak dependencies: 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:20:57.347 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:20:57.348 INFO:teuthology.orchestra.run.trial038.stdout:Install 32 Packages 2026-02-23T10:20:57.348 INFO:teuthology.orchestra.run.trial038.stdout:Upgrade 2 Packages 2026-02-23T10:20:57.348 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:20:57.348 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 60 M 2026-02-23T10:20:57.348 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:20:59.211 INFO:teuthology.orchestra.run.trial153.stdout:(1/34): ceph-base-20.3.0-5439.ge371433f.el9.x86 1.8 MB/s | 5.1 MB 00:02 2026-02-23T10:20:59.277 INFO:teuthology.orchestra.run.trial153.stdout:(2/34): ceph-selinux-20.3.0-5439.ge371433f.el9. 382 kB/s | 25 kB 00:00 2026-02-23T10:20:59.968 INFO:teuthology.orchestra.run.trial153.stdout:(3/34): libcephfs2-20.3.0-5439.ge371433f.el9.x8 1.2 MB/s | 860 kB 00:00 2026-02-23T10:21:00.151 INFO:teuthology.orchestra.run.trial153.stdout:(4/34): libradosstriper1-20.3.0-5439.ge371433f. 1.4 MB/s | 263 kB 00:00 2026-02-23T10:21:00.425 INFO:teuthology.orchestra.run.trial038.stdout:(1/34): ceph-base-20.3.0-5439.ge371433f.el9.x86 1.8 MB/s | 5.1 MB 00:02 2026-02-23T10:21:00.486 INFO:teuthology.orchestra.run.trial038.stdout:(2/34): ceph-selinux-20.3.0-5439.ge371433f.el9. 411 kB/s | 25 kB 00:00 2026-02-23T10:21:00.869 INFO:teuthology.orchestra.run.trial038.stdout:(3/34): libcephfs2-20.3.0-5439.ge371433f.el9.x8 2.2 MB/s | 860 kB 00:00 2026-02-23T10:21:01.005 INFO:teuthology.orchestra.run.trial038.stdout:(4/34): libradosstriper1-20.3.0-5439.ge371433f. 1.9 MB/s | 263 kB 00:00 2026-02-23T10:21:03.104 INFO:teuthology.orchestra.run.trial038.stdout:(5/34): ceph-radosgw-20.3.0-5439.ge371433f.el9. 2.0 MB/s | 11 MB 00:05 2026-02-23T10:21:03.108 INFO:teuthology.orchestra.run.trial153.stdout:(5/34): ceph-radosgw-20.3.0-5439.ge371433f.el9. 1.7 MB/s | 11 MB 00:06 2026-02-23T10:21:03.168 INFO:teuthology.orchestra.run.trial038.stdout:(6/34): python3-ceph-argparse-20.3.0-5439.ge371 701 kB/s | 45 kB 00:00 2026-02-23T10:21:03.174 INFO:teuthology.orchestra.run.trial153.stdout:(6/34): python3-ceph-argparse-20.3.0-5439.ge371 694 kB/s | 45 kB 00:00 2026-02-23T10:21:03.302 INFO:teuthology.orchestra.run.trial038.stdout:(7/34): python3-ceph-common-20.3.0-5439.ge37143 1.4 MB/s | 186 kB 00:00 2026-02-23T10:21:03.376 INFO:teuthology.orchestra.run.trial153.stdout:(7/34): python3-ceph-common-20.3.0-5439.ge37143 921 kB/s | 186 kB 00:00 2026-02-23T10:21:03.385 INFO:teuthology.orchestra.run.trial153.stdout:(8/34): librgw2-20.3.0-5439.ge371433f.el9.x86_6 1.7 MB/s | 5.6 MB 00:03 2026-02-23T10:21:03.439 INFO:teuthology.orchestra.run.trial038.stdout:(8/34): python3-cephfs-20.3.0-5439.ge371433f.el 1.5 MB/s | 204 kB 00:00 2026-02-23T10:21:03.582 INFO:teuthology.orchestra.run.trial153.stdout:(9/34): python3-cephfs-20.3.0-5439.ge371433f.el 990 kB/s | 204 kB 00:00 2026-02-23T10:21:03.592 INFO:teuthology.orchestra.run.trial153.stdout:(10/34): python3-rados-20.3.0-5439.ge371433f.el 1.5 MB/s | 324 kB 00:00 2026-02-23T10:21:03.599 INFO:teuthology.orchestra.run.trial038.stdout:(9/34): librgw2-20.3.0-5439.ge371433f.el9.x86_6 2.2 MB/s | 5.6 MB 00:02 2026-02-23T10:21:03.642 INFO:teuthology.orchestra.run.trial038.stdout:(10/34): python3-rados-20.3.0-5439.ge371433f.el 1.6 MB/s | 324 kB 00:00 2026-02-23T10:21:03.655 INFO:teuthology.orchestra.run.trial153.stdout:(11/34): python3-rgw-20.3.0-5439.ge371433f.el9. 1.6 MB/s | 100 kB 00:00 2026-02-23T10:21:03.708 INFO:teuthology.orchestra.run.trial038.stdout:(11/34): python3-rgw-20.3.0-5439.ge371433f.el9. 1.5 MB/s | 100 kB 00:00 2026-02-23T10:21:03.712 INFO:teuthology.orchestra.run.trial153.stdout:(12/34): mailcap-2.1.49-5.el9.noarch.rpm 590 kB/s | 33 kB 00:00 2026-02-23T10:21:03.750 INFO:teuthology.orchestra.run.trial153.stdout:(13/34): pciutils-3.7.0-7.el9.x86_64.rpm 2.4 MB/s | 93 kB 00:00 2026-02-23T10:21:03.789 INFO:teuthology.orchestra.run.trial038.stdout:(12/34): python3-rbd-20.3.0-5439.ge371433f.el9. 1.6 MB/s | 305 kB 00:00 2026-02-23T10:21:03.835 INFO:teuthology.orchestra.run.trial153.stdout:(14/34): python3-rbd-20.3.0-5439.ge371433f.el9. 1.2 MB/s | 305 kB 00:00 2026-02-23T10:21:03.902 INFO:teuthology.orchestra.run.trial153.stdout:(15/34): boost-program-options-1.75.0-13.el9.x8 686 kB/s | 104 kB 00:00 2026-02-23T10:21:03.957 INFO:teuthology.orchestra.run.trial153.stdout:(16/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.8 MB/s | 160 kB 00:00 2026-02-23T10:21:03.974 INFO:teuthology.orchestra.run.trial038.stdout:(13/34): mailcap-2.1.49-5.el9.noarch.rpm 125 kB/s | 33 kB 00:00 2026-02-23T10:21:03.975 INFO:teuthology.orchestra.run.trial153.stdout:(17/34): libnbd-1.20.3-4.el9.x86_64.rpm 1.1 MB/s | 164 kB 00:00 2026-02-23T10:21:03.984 INFO:teuthology.orchestra.run.trial153.stdout:(18/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.6 MB/s | 45 kB 00:00 2026-02-23T10:21:04.029 INFO:teuthology.orchestra.run.trial153.stdout:(19/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.4 MB/s | 292 kB 00:00 2026-02-23T10:21:04.037 INFO:teuthology.orchestra.run.trial038.stdout:(14/34): boost-program-options-1.75.0-13.el9.x8 1.6 MB/s | 104 kB 00:00 2026-02-23T10:21:04.060 INFO:teuthology.orchestra.run.trial153.stdout:(20/34): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2026-02-23T10:21:04.066 INFO:teuthology.orchestra.run.trial153.stdout:(21/34): qatlib-25.08.0-2.el9.x86_64.rpm 6.4 MB/s | 240 kB 00:00 2026-02-23T10:21:04.081 INFO:teuthology.orchestra.run.trial038.stdout:(15/34): libnbd-1.20.3-4.el9.x86_64.rpm 3.7 MB/s | 164 kB 00:00 2026-02-23T10:21:04.085 INFO:teuthology.orchestra.run.trial153.stdout:(22/34): qatlib-service-25.08.0-2.el9.x86_64.rp 1.4 MB/s | 37 kB 00:00 2026-02-23T10:21:04.093 INFO:teuthology.orchestra.run.trial153.stdout:(23/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.4 MB/s | 66 kB 00:00 2026-02-23T10:21:04.105 INFO:teuthology.orchestra.run.trial038.stdout:(16/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.5 MB/s | 160 kB 00:00 2026-02-23T10:21:04.127 INFO:teuthology.orchestra.run.trial038.stdout:(17/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 2.0 MB/s | 45 kB 00:00 2026-02-23T10:21:04.165 INFO:teuthology.orchestra.run.trial153.stdout:(24/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.8 MB/s | 308 kB 00:00 2026-02-23T10:21:04.171 INFO:teuthology.orchestra.run.trial038.stdout:(18/34): librdkafka-1.6.1-102.el9.x86_64.rpm 15 MB/s | 662 kB 00:00 2026-02-23T10:21:04.174 INFO:teuthology.orchestra.run.trial153.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 2.6 MB/s | 25 kB 00:00 2026-02-23T10:21:04.176 INFO:teuthology.orchestra.run.trial038.stdout:(19/34): pciutils-3.7.0-7.el9.x86_64.rpm 240 kB/s | 93 kB 00:00 2026-02-23T10:21:04.183 INFO:teuthology.orchestra.run.trial153.stdout:(26/34): liboath-2.6.12-1.el9.x86_64.rpm 5.0 MB/s | 49 kB 00:00 2026-02-23T10:21:04.192 INFO:teuthology.orchestra.run.trial153.stdout:(27/34): libarrow-9.0.0-15.el9.x86_64.rpm 45 MB/s | 4.4 MB 00:00 2026-02-23T10:21:04.194 INFO:teuthology.orchestra.run.trial153.stdout:(28/34): libunwind-1.6.2-1.el9.x86_64.rpm 6.5 MB/s | 67 kB 00:00 2026-02-23T10:21:04.195 INFO:teuthology.orchestra.run.trial038.stdout:(20/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 12 MB/s | 292 kB 00:00 2026-02-23T10:21:04.217 INFO:teuthology.orchestra.run.trial153.stdout:(29/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 33 MB/s | 838 kB 00:00 2026-02-23T10:21:04.217 INFO:teuthology.orchestra.run.trial038.stdout:(21/34): qatlib-service-25.08.0-2.el9.x86_64.rp 1.7 MB/s | 37 kB 00:00 2026-02-23T10:21:04.218 INFO:teuthology.orchestra.run.trial153.stdout:(30/34): re2-20211101-20.el9.x86_64.rpm 7.6 MB/s | 191 kB 00:00 2026-02-23T10:21:04.237 INFO:teuthology.orchestra.run.trial153.stdout:(31/34): thrift-0.15.0-4.el9.x86_64.rpm 78 MB/s | 1.6 MB 00:00 2026-02-23T10:21:04.239 INFO:teuthology.orchestra.run.trial038.stdout:(22/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 3.0 MB/s | 66 kB 00:00 2026-02-23T10:21:04.261 INFO:teuthology.orchestra.run.trial038.stdout:(23/34): qatlib-25.08.0-2.el9.x86_64.rpm 2.8 MB/s | 240 kB 00:00 2026-02-23T10:21:04.308 INFO:teuthology.orchestra.run.trial038.stdout:(24/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.4 MB/s | 308 kB 00:00 2026-02-23T10:21:04.318 INFO:teuthology.orchestra.run.trial038.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 2.4 MB/s | 25 kB 00:00 2026-02-23T10:21:04.329 INFO:teuthology.orchestra.run.trial038.stdout:(26/34): liboath-2.6.12-1.el9.x86_64.rpm 4.5 MB/s | 49 kB 00:00 2026-02-23T10:21:04.340 INFO:teuthology.orchestra.run.trial038.stdout:(27/34): libunwind-1.6.2-1.el9.x86_64.rpm 6.1 MB/s | 67 kB 00:00 2026-02-23T10:21:04.364 INFO:teuthology.orchestra.run.trial038.stdout:(28/34): libarrow-9.0.0-15.el9.x86_64.rpm 43 MB/s | 4.4 MB 00:00 2026-02-23T10:21:04.367 INFO:teuthology.orchestra.run.trial038.stdout:(29/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 30 MB/s | 838 kB 00:00 2026-02-23T10:21:04.389 INFO:teuthology.orchestra.run.trial038.stdout:(30/34): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2026-02-23T10:21:04.419 INFO:teuthology.orchestra.run.trial038.stdout:(31/34): thrift-0.15.0-4.el9.x86_64.rpm 31 MB/s | 1.6 MB 00:00 2026-02-23T10:21:06.281 INFO:teuthology.orchestra.run.trial153.stdout:(32/34): librbd1-20.3.0-5439.ge371433f.el9.x86_ 1.3 MB/s | 2.7 MB 00:02 2026-02-23T10:21:06.634 INFO:teuthology.orchestra.run.trial153.stdout:(33/34): librados2-20.3.0-5439.ge371433f.el9.x8 1.4 MB/s | 3.3 MB 00:02 2026-02-23T10:21:06.691 INFO:teuthology.orchestra.run.trial153.stdout:(34/34): ceph-common-20.3.0-5439.ge371433f.el9. 2.0 MB/s | 21 MB 00:10 2026-02-23T10:21:06.694 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:21:06.695 INFO:teuthology.orchestra.run.trial153.stdout:Total 5.6 MB/s | 60 MB 00:10 2026-02-23T10:21:06.782 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:21:06.800 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:21:06.800 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:21:06.919 INFO:teuthology.orchestra.run.trial038.stdout:(32/34): librados2-20.3.0-5439.ge371433f.el9.x8 1.3 MB/s | 3.3 MB 00:02 2026-02-23T10:21:07.045 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:21:07.046 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:21:07.328 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:21:07.470 INFO:teuthology.orchestra.run.trial153.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-23T10:21:07.475 INFO:teuthology.orchestra.run.trial153.stdout: Upgrading : librados2-2:20.3.0-5439.ge371433f.el9.x86_64 2/36 2026-02-23T10:21:07.506 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: librados2-2:20.3.0-5439.ge371433f.el9.x86_64 2/36 2026-02-23T10:21:07.543 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-rados-2:20.3.0-5439.ge371433f.el9.x86_64 3/36 2026-02-23T10:21:07.544 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 4/36 2026-02-23T10:21:07.556 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 4/36 2026-02-23T10:21:07.564 INFO:teuthology.orchestra.run.trial153.stdout: Installing : liboath-2.6.12-1.el9.x86_64 5/36 2026-02-23T10:21:07.567 INFO:teuthology.orchestra.run.trial153.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/36 2026-02-23T10:21:07.570 INFO:teuthology.orchestra.run.trial153.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/36 2026-02-23T10:21:07.574 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 8/36 2026-02-23T10:21:07.585 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-ceph-argparse-2:20.3.0-5439.ge371433f.el9. 9/36 2026-02-23T10:21:07.597 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-cephfs-2:20.3.0-5439.ge371433f.el9.x86_64 10/36 2026-02-23T10:21:07.598 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libradosstriper1-2:20.3.0-5439.ge371433f.el9.x86_6 11/36 2026-02-23T10:21:07.618 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5439.ge371433f.el9.x86_6 11/36 2026-02-23T10:21:07.622 INFO:teuthology.orchestra.run.trial153.stdout: Installing : thrift-0.15.0-4.el9.x86_64 12/36 2026-02-23T10:21:07.652 INFO:teuthology.orchestra.run.trial153.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-23T10:21:07.656 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-23T10:21:07.662 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-23T10:21:07.664 INFO:teuthology.orchestra.run.trial153.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-23T10:21:07.669 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-23T10:21:07.783 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-23T10:21:07.784 INFO:teuthology.orchestra.run.trial153.stdout: Upgrading : librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 19/36 2026-02-23T10:21:07.809 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 19/36 2026-02-23T10:21:07.812 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-rbd-2:20.3.0-5439.ge371433f.el9.x86_64 20/36 2026-02-23T10:21:07.820 INFO:teuthology.orchestra.run.trial153.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-23T10:21:08.069 INFO:teuthology.orchestra.run.trial153.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-23T10:21:08.070 INFO:teuthology.orchestra.run.trial153.stdout: Installing : librgw2-2:20.3.0-5439.ge371433f.el9.x86_64 23/36 2026-02-23T10:21:08.087 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: librgw2-2:20.3.0-5439.ge371433f.el9.x86_64 23/36 2026-02-23T10:21:08.091 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-rgw-2:20.3.0-5439.ge371433f.el9.x86_64 24/36 2026-02-23T10:21:08.095 INFO:teuthology.orchestra.run.trial153.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-23T10:21:08.098 INFO:teuthology.orchestra.run.trial153.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-23T10:21:08.099 INFO:teuthology.orchestra.run.trial153.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-23T10:21:08.117 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-23T10:21:08.120 INFO:teuthology.orchestra.run.trial153.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-23T10:21:08.154 INFO:teuthology.orchestra.run.trial153.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-23T10:21:08.156 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-ceph-common-2:20.3.0-5439.ge371433f.el9.x8 30/36 2026-02-23T10:21:08.293 INFO:teuthology.orchestra.run.trial038.stdout:(33/34): librbd1-20.3.0-5439.ge371433f.el9.x86_ 711 kB/s | 2.7 MB 00:03 2026-02-23T10:21:09.067 INFO:teuthology.orchestra.run.trial038.stdout:(34/34): ceph-common-20.3.0-5439.ge371433f.el9. 1.8 MB/s | 21 MB 00:11 2026-02-23T10:21:09.069 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:21:09.069 INFO:teuthology.orchestra.run.trial038.stdout:Total 5.1 MB/s | 60 MB 00:11 2026-02-23T10:21:09.095 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 31/36 2026-02-23T10:21:09.097 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 31/36 2026-02-23T10:21:09.160 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:21:09.177 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:21:09.177 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:21:09.362 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 31/36 2026-02-23T10:21:09.366 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 32/36 2026-02-23T10:21:09.405 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 32/36 2026-02-23T10:21:09.405 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-23T10:21:09.405 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-23T10:21:09.405 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:21:09.408 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-selinux-2:20.3.0-5439.ge371433f.el9.x86_64 33/36 2026-02-23T10:21:09.422 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:21:09.422 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:21:09.703 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:21:09.852 INFO:teuthology.orchestra.run.trial038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-23T10:21:09.857 INFO:teuthology.orchestra.run.trial038.stdout: Upgrading : librados2-2:20.3.0-5439.ge371433f.el9.x86_64 2/36 2026-02-23T10:21:09.888 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librados2-2:20.3.0-5439.ge371433f.el9.x86_64 2/36 2026-02-23T10:21:09.926 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rados-2:20.3.0-5439.ge371433f.el9.x86_64 3/36 2026-02-23T10:21:09.927 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 4/36 2026-02-23T10:21:09.939 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 4/36 2026-02-23T10:21:09.947 INFO:teuthology.orchestra.run.trial038.stdout: Installing : liboath-2.6.12-1.el9.x86_64 5/36 2026-02-23T10:21:09.949 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/36 2026-02-23T10:21:09.952 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/36 2026-02-23T10:21:09.956 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 8/36 2026-02-23T10:21:09.968 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-ceph-argparse-2:20.3.0-5439.ge371433f.el9. 9/36 2026-02-23T10:21:09.981 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cephfs-2:20.3.0-5439.ge371433f.el9.x86_64 10/36 2026-02-23T10:21:09.982 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libradosstriper1-2:20.3.0-5439.ge371433f.el9.x86_6 11/36 2026-02-23T10:21:10.007 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5439.ge371433f.el9.x86_6 11/36 2026-02-23T10:21:10.011 INFO:teuthology.orchestra.run.trial038.stdout: Installing : thrift-0.15.0-4.el9.x86_64 12/36 2026-02-23T10:21:10.043 INFO:teuthology.orchestra.run.trial038.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-23T10:21:10.047 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-23T10:21:10.053 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-23T10:21:10.055 INFO:teuthology.orchestra.run.trial038.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-23T10:21:10.060 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-23T10:21:10.179 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-23T10:21:10.180 INFO:teuthology.orchestra.run.trial038.stdout: Upgrading : librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 19/36 2026-02-23T10:21:10.208 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 19/36 2026-02-23T10:21:10.212 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rbd-2:20.3.0-5439.ge371433f.el9.x86_64 20/36 2026-02-23T10:21:10.219 INFO:teuthology.orchestra.run.trial038.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-23T10:21:10.482 INFO:teuthology.orchestra.run.trial038.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-23T10:21:10.483 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librgw2-2:20.3.0-5439.ge371433f.el9.x86_64 23/36 2026-02-23T10:21:10.501 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librgw2-2:20.3.0-5439.ge371433f.el9.x86_64 23/36 2026-02-23T10:21:10.506 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rgw-2:20.3.0-5439.ge371433f.el9.x86_64 24/36 2026-02-23T10:21:10.511 INFO:teuthology.orchestra.run.trial038.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-23T10:21:10.513 INFO:teuthology.orchestra.run.trial038.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-23T10:21:10.514 INFO:teuthology.orchestra.run.trial038.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-23T10:21:10.532 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-23T10:21:10.535 INFO:teuthology.orchestra.run.trial038.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-23T10:21:10.568 INFO:teuthology.orchestra.run.trial038.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-23T10:21:10.570 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-ceph-common-2:20.3.0-5439.ge371433f.el9.x8 30/36 2026-02-23T10:21:11.556 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 31/36 2026-02-23T10:21:11.558 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 31/36 2026-02-23T10:21:11.832 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 31/36 2026-02-23T10:21:11.835 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 32/36 2026-02-23T10:21:11.876 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 32/36 2026-02-23T10:21:11.876 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-23T10:21:11.876 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-23T10:21:11.876 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:21:11.879 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-selinux-2:20.3.0-5439.ge371433f.el9.x86_64 33/36 2026-02-23T10:21:13.905 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5439.ge371433f.el9.x86_64 33/36 2026-02-23T10:21:13.905 INFO:teuthology.orchestra.run.trial153.stdout:skipping the directory /sys 2026-02-23T10:21:13.905 INFO:teuthology.orchestra.run.trial153.stdout:skipping the directory /proc 2026-02-23T10:21:13.905 INFO:teuthology.orchestra.run.trial153.stdout:skipping the directory /mnt 2026-02-23T10:21:13.905 INFO:teuthology.orchestra.run.trial153.stdout:skipping the directory /var/tmp 2026-02-23T10:21:13.905 INFO:teuthology.orchestra.run.trial153.stdout:skipping the directory /home 2026-02-23T10:21:13.906 INFO:teuthology.orchestra.run.trial153.stdout:skipping the directory /root 2026-02-23T10:21:13.906 INFO:teuthology.orchestra.run.trial153.stdout:skipping the directory /tmp 2026-02-23T10:21:13.906 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:21:14.396 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-radosgw-2:20.3.0-5439.ge371433f.el9.x86_64 34/36 2026-02-23T10:21:14.420 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5439.ge371433f.el9.x86_64 34/36 2026-02-23T10:21:14.420 INFO:teuthology.orchestra.run.trial153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:21:14.421 INFO:teuthology.orchestra.run.trial153.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-23T10:21:14.421 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-23T10:21:14.421 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-23T10:21:14.421 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:21:14.422 INFO:teuthology.orchestra.run.trial153.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-23T10:21:14.435 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-23T10:21:14.435 INFO:teuthology.orchestra.run.trial153.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-23T10:21:14.490 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-23T10:21:14.490 INFO:teuthology.orchestra.run.trial153.stdout:Creating group 'qat' with GID 992. 2026-02-23T10:21:14.490 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:21:14.782 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 1/36 2026-02-23T10:21:14.783 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 2/36 2026-02-23T10:21:14.783 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-radosgw-2:20.3.0-5439.ge371433f.el9.x86_64 3/36 2026-02-23T10:21:14.783 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-selinux-2:20.3.0-5439.ge371433f.el9.x86_64 4/36 2026-02-23T10:21:14.783 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 5/36 2026-02-23T10:21:14.783 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libradosstriper1-2:20.3.0-5439.ge371433f.el9.x86_6 6/36 2026-02-23T10:21:14.783 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : librgw2-2:20.3.0-5439.ge371433f.el9.x86_64 7/36 2026-02-23T10:21:14.783 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5439.ge371433f.el9. 8/36 2026-02-23T10:21:14.783 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-ceph-common-2:20.3.0-5439.ge371433f.el9.x8 9/36 2026-02-23T10:21:14.783 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-cephfs-2:20.3.0-5439.ge371433f.el9.x86_64 10/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-rados-2:20.3.0-5439.ge371433f.el9.x86_64 11/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-rbd-2:20.3.0-5439.ge371433f.el9.x86_64 12/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-rgw-2:20.3.0-5439.ge371433f.el9.x86_64 13/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-23T10:21:14.784 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : librados2-2:20.3.0-5439.ge371433f.el9.x86_64 33/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-23T10:21:14.785 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 35/36 2026-02-23T10:21:14.842 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout:Upgraded: 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout: librados2-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout: librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout: ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout: ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout: ceph-radosgw-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout: ceph-selinux-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-23T10:21:14.843 INFO:teuthology.orchestra.run.trial153.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: libradosstriper1-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: librgw2-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-23T10:21:14.844 INFO:teuthology.orchestra.run.trial153.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: python3-ceph-argparse-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: python3-ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: python3-cephfs-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: python3-rados-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: python3-rbd-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: python3-rgw-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: re2-1:20211101-20.el9.x86_64 2026-02-23T10:21:14.845 INFO:teuthology.orchestra.run.trial153.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-23T10:21:14.846 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:21:14.846 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:21:14.882 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-test 2026-02-23T10:21:15.196 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:21 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:21:15.278 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repository Size 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: ceph-test x86_64 2:20.3.0-5439.ge371433f.el9 ceph 126 M 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout:Installing dependencies: 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: libcephsqlite x86_64 2:20.3.0-5439.ge371433f.el9 ceph 153 k 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout:Install 12 Packages 2026-02-23T10:21:15.279 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:21:15.280 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 131 M 2026-02-23T10:21:15.280 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 977 M 2026-02-23T10:21:15.280 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:21:15.830 INFO:teuthology.orchestra.run.trial153.stdout:(1/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 27 MB/s | 2.1 MB 00:00 2026-02-23T10:21:15.859 INFO:teuthology.orchestra.run.trial153.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 800 kB/s | 23 kB 00:00 2026-02-23T10:21:15.883 INFO:teuthology.orchestra.run.trial153.stdout:(3/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 3.9 MB/s | 94 kB 00:00 2026-02-23T10:21:15.951 INFO:teuthology.orchestra.run.trial153.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 7.6 MB/s | 525 kB 00:00 2026-02-23T10:21:15.963 INFO:teuthology.orchestra.run.trial153.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 15 MB/s | 176 kB 00:00 2026-02-23T10:21:15.975 INFO:teuthology.orchestra.run.trial153.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 19 MB/s | 233 kB 00:00 2026-02-23T10:21:16.005 INFO:teuthology.orchestra.run.trial153.stdout:(7/12): protobuf-3.14.0-17.el9.x86_64.rpm 34 MB/s | 1.0 MB 00:00 2026-02-23T10:21:16.017 INFO:teuthology.orchestra.run.trial153.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 24 MB/s | 303 kB 00:00 2026-02-23T10:21:16.027 INFO:teuthology.orchestra.run.trial153.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 6.0 MB/s | 64 kB 00:00 2026-02-23T10:21:16.174 INFO:teuthology.orchestra.run.trial153.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 828 kB/s | 122 kB 00:00 2026-02-23T10:21:16.243 INFO:teuthology.orchestra.run.trial153.stdout:(11/12): libcephsqlite-20.3.0-5439.ge371433f.el 311 kB/s | 153 kB 00:00 2026-02-23T10:21:16.345 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5439.ge371433f.el9.x86_64 33/36 2026-02-23T10:21:16.345 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /sys 2026-02-23T10:21:16.345 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /proc 2026-02-23T10:21:16.345 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /mnt 2026-02-23T10:21:16.345 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /var/tmp 2026-02-23T10:21:16.345 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /home 2026-02-23T10:21:16.345 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /root 2026-02-23T10:21:16.345 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /tmp 2026-02-23T10:21:16.345 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:21:16.853 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-radosgw-2:20.3.0-5439.ge371433f.el9.x86_64 34/36 2026-02-23T10:21:16.876 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5439.ge371433f.el9.x86_64 34/36 2026-02-23T10:21:16.877 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:21:16.877 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-23T10:21:16.877 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-23T10:21:16.877 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-23T10:21:16.877 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:21:16.879 INFO:teuthology.orchestra.run.trial038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-23T10:21:16.892 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-23T10:21:16.892 INFO:teuthology.orchestra.run.trial038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-23T10:21:16.947 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-23T10:21:16.948 INFO:teuthology.orchestra.run.trial038.stdout:Creating group 'qat' with GID 992. 2026-02-23T10:21:16.948 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:21:17.250 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 1/36 2026-02-23T10:21:17.250 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 2/36 2026-02-23T10:21:17.250 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-radosgw-2:20.3.0-5439.ge371433f.el9.x86_64 3/36 2026-02-23T10:21:17.250 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-selinux-2:20.3.0-5439.ge371433f.el9.x86_64 4/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 5/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libradosstriper1-2:20.3.0-5439.ge371433f.el9.x86_6 6/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librgw2-2:20.3.0-5439.ge371433f.el9.x86_64 7/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5439.ge371433f.el9. 8/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-ceph-common-2:20.3.0-5439.ge371433f.el9.x8 9/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cephfs-2:20.3.0-5439.ge371433f.el9.x86_64 10/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rados-2:20.3.0-5439.ge371433f.el9.x86_64 11/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rbd-2:20.3.0-5439.ge371433f.el9.x86_64 12/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rgw-2:20.3.0-5439.ge371433f.el9.x86_64 13/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-23T10:21:17.251 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-23T10:21:17.252 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-23T10:21:17.253 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-23T10:21:17.253 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-23T10:21:17.253 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-23T10:21:17.253 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librados2-2:20.3.0-5439.ge371433f.el9.x86_64 33/36 2026-02-23T10:21:17.253 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-23T10:21:17.253 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 35/36 2026-02-23T10:21:17.310 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-23T10:21:17.311 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:21:17.311 INFO:teuthology.orchestra.run.trial038.stdout:Upgraded: 2026-02-23T10:21:17.311 INFO:teuthology.orchestra.run.trial038.stdout: librados2-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.311 INFO:teuthology.orchestra.run.trial038.stdout: librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.311 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:21:17.311 INFO:teuthology.orchestra.run.trial038.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-23T10:21:17.311 INFO:teuthology.orchestra.run.trial038.stdout: ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.311 INFO:teuthology.orchestra.run.trial038.stdout: ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.311 INFO:teuthology.orchestra.run.trial038.stdout: ceph-radosgw-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.311 INFO:teuthology.orchestra.run.trial038.stdout: ceph-selinux-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: libradosstriper1-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: librgw2-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-23T10:21:17.312 INFO:teuthology.orchestra.run.trial038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-argparse-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-common-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: python3-cephfs-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: python3-rados-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: python3-rbd-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: python3-rgw-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-23T10:21:17.313 INFO:teuthology.orchestra.run.trial038.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-23T10:21:17.314 INFO:teuthology.orchestra.run.trial038.stdout: re2-1:20211101-20.el9.x86_64 2026-02-23T10:21:17.314 INFO:teuthology.orchestra.run.trial038.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-23T10:21:17.314 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:21:17.314 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:21:17.355 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-test 2026-02-23T10:21:17.664 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:21 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:21:17.748 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:21:17.749 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:21:17.749 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-23T10:21:17.749 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:21:17.749 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:21:17.750 INFO:teuthology.orchestra.run.trial038.stdout: ceph-test x86_64 2:20.3.0-5439.ge371433f.el9 ceph 126 M 2026-02-23T10:21:17.750 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-23T10:21:17.750 INFO:teuthology.orchestra.run.trial038.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-23T10:21:17.750 INFO:teuthology.orchestra.run.trial038.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-23T10:21:17.750 INFO:teuthology.orchestra.run.trial038.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-23T10:21:17.750 INFO:teuthology.orchestra.run.trial038.stdout: libcephsqlite x86_64 2:20.3.0-5439.ge371433f.el9 ceph 153 k 2026-02-23T10:21:17.750 INFO:teuthology.orchestra.run.trial038.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-23T10:21:17.750 INFO:teuthology.orchestra.run.trial038.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-23T10:21:17.750 INFO:teuthology.orchestra.run.trial038.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-23T10:21:17.750 INFO:teuthology.orchestra.run.trial038.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-23T10:21:17.750 INFO:teuthology.orchestra.run.trial038.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-23T10:21:17.751 INFO:teuthology.orchestra.run.trial038.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-23T10:21:17.751 INFO:teuthology.orchestra.run.trial038.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-23T10:21:17.751 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:21:17.751 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:21:17.751 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:21:17.751 INFO:teuthology.orchestra.run.trial038.stdout:Install 12 Packages 2026-02-23T10:21:17.751 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:21:17.751 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 131 M 2026-02-23T10:21:17.751 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 977 M 2026-02-23T10:21:17.751 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:21:18.653 INFO:teuthology.orchestra.run.trial038.stdout:(1/12): libcephsqlite-20.3.0-5439.ge371433f.el9 329 kB/s | 153 kB 00:00 2026-02-23T10:21:19.073 INFO:teuthology.orchestra.run.trial038.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 54 kB/s | 23 kB 00:00 2026-02-23T10:21:19.330 INFO:teuthology.orchestra.run.trial038.stdout:(3/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 366 kB/s | 94 kB 00:00 2026-02-23T10:21:19.801 INFO:teuthology.orchestra.run.trial038.stdout:(4/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 1.3 MB/s | 2.1 MB 00:01 2026-02-23T10:21:20.184 INFO:teuthology.orchestra.run.trial038.stdout:(5/12): libasan8-15.2.1-7.el9.x86_64.rpm 615 kB/s | 525 kB 00:00 2026-02-23T10:21:20.532 INFO:teuthology.orchestra.run.trial038.stdout:(6/12): libubsan-11.5.0-14.el9.x86_64.rpm 241 kB/s | 176 kB 00:00 2026-02-23T10:21:20.954 INFO:teuthology.orchestra.run.trial038.stdout:(7/12): libxslt-1.1.34-12.el9.x86_64.rpm 302 kB/s | 233 kB 00:00 2026-02-23T10:21:21.469 INFO:teuthology.orchestra.run.trial038.stdout:(8/12): protobuf-3.14.0-17.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2026-02-23T10:21:21.743 INFO:teuthology.orchestra.run.trial038.stdout:(9/12): socat-1.7.4.1-8.el9.x86_64.rpm 384 kB/s | 303 kB 00:00 2026-02-23T10:21:21.870 INFO:teuthology.orchestra.run.trial038.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 965 kB/s | 122 kB 00:00 2026-02-23T10:21:22.089 INFO:teuthology.orchestra.run.trial038.stdout:(11/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 103 kB/s | 64 kB 00:00 2026-02-23T10:22:16.461 INFO:teuthology.orchestra.run.trial038.stdout:(12/12): ceph-test-20.3.0-5439.ge371433f.el9.x8 2.2 MB/s | 126 MB 00:58 2026-02-23T10:22:16.463 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:22:16.463 INFO:teuthology.orchestra.run.trial038.stdout:Total 2.2 MB/s | 131 MB 00:58 2026-02-23T10:22:16.538 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:22:16.550 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:22:16.550 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:22:16.978 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:22:16.978 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:22:17.434 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:22:17.442 INFO:teuthology.orchestra.run.trial038.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-23T10:22:17.451 INFO:teuthology.orchestra.run.trial038.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-23T10:22:17.457 INFO:teuthology.orchestra.run.trial038.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-23T10:22:17.462 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-23T10:22:17.465 INFO:teuthology.orchestra.run.trial038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-23T10:22:17.469 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-23T10:22:17.470 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-23T10:22:17.488 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-23T10:22:17.490 INFO:teuthology.orchestra.run.trial038.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-23T10:22:17.501 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-23T10:22:17.510 INFO:teuthology.orchestra.run.trial038.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-23T10:22:17.511 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephsqlite-2:20.3.0-5439.ge371433f.el9.x86_64 11/12 2026-02-23T10:22:24.431 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5439.ge371433f.el9.x86_64 11/12 2026-02-23T10:22:24.449 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-test-2:20.3.0-5439.ge371433f.el9.x86_64 12/12 2026-02-23T10:22:25.224 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-test-2:20.3.0-5439.ge371433f.el9.x86_64 12/12 2026-02-23T10:22:25.224 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-test-2:20.3.0-5439.ge371433f.el9.x86_64 1/12 2026-02-23T10:22:25.224 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephsqlite-2:20.3.0-5439.ge371433f.el9.x86_64 2/12 2026-02-23T10:22:25.224 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-23T10:22:25.224 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-23T10:22:25.224 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-23T10:22:25.224 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-23T10:22:25.224 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-23T10:22:25.224 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-23T10:22:25.224 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-23T10:22:25.225 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-23T10:22:25.225 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-23T10:22:25.278 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-23T10:22:25.278 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:22:25.278 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:22:25.278 INFO:teuthology.orchestra.run.trial038.stdout: ceph-test-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:22:25.278 INFO:teuthology.orchestra.run.trial038.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-23T10:22:25.278 INFO:teuthology.orchestra.run.trial038.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-23T10:22:25.278 INFO:teuthology.orchestra.run.trial038.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-23T10:22:25.278 INFO:teuthology.orchestra.run.trial038.stdout: libcephsqlite-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:22:25.278 INFO:teuthology.orchestra.run.trial038.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-23T10:22:25.279 INFO:teuthology.orchestra.run.trial038.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-23T10:22:25.279 INFO:teuthology.orchestra.run.trial038.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-23T10:22:25.279 INFO:teuthology.orchestra.run.trial038.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-23T10:22:25.279 INFO:teuthology.orchestra.run.trial038.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-23T10:22:25.279 INFO:teuthology.orchestra.run.trial038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-23T10:22:25.279 INFO:teuthology.orchestra.run.trial038.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-23T10:22:25.279 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:22:25.279 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:22:25.319 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph 2026-02-23T10:22:25.624 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:29 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:22:25.707 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:22:25.709 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:22:25.709 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-23T10:22:25.709 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:22:25.709 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout: ceph x86_64 2:20.3.0-5439.ge371433f.el9 ceph 6.5 k 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mds x86_64 2:20.3.0-5439.ge371433f.el9 ceph 2.0 M 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr x86_64 2:20.3.0-5439.ge371433f.el9 ceph 937 k 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 298 k 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mon x86_64 2:20.3.0-5439.ge371433f.el9 ceph 4.9 M 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd x86_64 2:20.3.0-5439.ge371433f.el9 ceph 164 k 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-classic x86_64 2:20.3.0-5439.ge371433f.el9 ceph 15 M 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-crimson x86_64 2:20.3.0-5439.ge371433f.el9 ceph 87 M 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-23T10:22:25.710 INFO:teuthology.orchestra.run.trial038.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-23T10:22:25.711 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-collections 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: noarch 3.0.0-8.el9 epel 23 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-23T10:22:25.712 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout:Install 42 Packages 2026-02-23T10:22:25.713 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:22:25.714 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 114 M 2026-02-23T10:22:25.714 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 961 M 2026-02-23T10:22:25.714 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:22:26.308 INFO:teuthology.orchestra.run.trial038.stdout:(1/42): ceph-20.3.0-5439.ge371433f.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2026-02-23T10:22:27.018 INFO:teuthology.orchestra.run.trial038.stdout:(2/42): ceph-mgr-20.3.0-5439.ge371433f.el9.x86_ 933 kB/s | 937 kB 00:01 2026-02-23T10:22:27.154 INFO:teuthology.orchestra.run.trial038.stdout:(3/42): ceph-osd-20.3.0-5439.ge371433f.el9.x86_ 1.2 MB/s | 164 kB 00:00 2026-02-23T10:22:27.528 INFO:teuthology.orchestra.run.trial038.stdout:(4/42): ceph-mds-20.3.0-5439.ge371433f.el9.x86_ 1.3 MB/s | 2.0 MB 00:01 2026-02-23T10:22:27.763 INFO:teuthology.orchestra.run.trial038.stdout:(5/42): ceph-mon-20.3.0-5439.ge371433f.el9.x86_ 3.3 MB/s | 4.9 MB 00:01 2026-02-23T10:22:27.892 INFO:teuthology.orchestra.run.trial038.stdout:(6/42): ceph-mgr-modules-core-20.3.0-5439.ge371 2.3 MB/s | 298 kB 00:00 2026-02-23T10:22:27.923 INFO:teuthology.orchestra.run.trial038.stdout:(7/42): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.3 MB/s | 40 kB 00:00 2026-02-23T10:22:27.941 INFO:teuthology.orchestra.run.trial038.stdout:(8/42): libconfig-1.7.2-9.el9.x86_64.rpm 3.9 MB/s | 72 kB 00:00 2026-02-23T10:22:27.962 INFO:teuthology.orchestra.run.trial038.stdout:(9/42): python3-cffi-1.14.5-5.el9.x86_64.rpm 12 MB/s | 253 kB 00:00 2026-02-23T10:22:28.001 INFO:teuthology.orchestra.run.trial038.stdout:(10/42): python3-cryptography-36.0.1-5.el9.x86_ 32 MB/s | 1.2 MB 00:00 2026-02-23T10:22:28.012 INFO:teuthology.orchestra.run.trial038.stdout:(11/42): python3-ply-3.11-14.el9.noarch.rpm 9.9 MB/s | 106 kB 00:00 2026-02-23T10:22:28.023 INFO:teuthology.orchestra.run.trial038.stdout:(12/42): python3-pycparser-2.20-6.el9.noarch.rp 12 MB/s | 135 kB 00:00 2026-02-23T10:22:28.035 INFO:teuthology.orchestra.run.trial038.stdout:(13/42): python3-pyparsing-2.4.7-9.el9.noarch.r 13 MB/s | 150 kB 00:00 2026-02-23T10:22:28.046 INFO:teuthology.orchestra.run.trial038.stdout:(14/42): python3-requests-2.25.1-10.el9.noarch. 11 MB/s | 126 kB 00:00 2026-02-23T10:22:28.059 INFO:teuthology.orchestra.run.trial038.stdout:(15/42): python3-urllib3-1.26.5-7.el9.noarch.rp 17 MB/s | 218 kB 00:00 2026-02-23T10:22:28.071 INFO:teuthology.orchestra.run.trial038.stdout:(16/42): unzip-6.0-59.el9.x86_64.rpm 14 MB/s | 182 kB 00:00 2026-02-23T10:22:28.082 INFO:teuthology.orchestra.run.trial038.stdout:(17/42): zip-3.0-35.el9.x86_64.rpm 24 MB/s | 266 kB 00:00 2026-02-23T10:22:28.810 INFO:teuthology.orchestra.run.trial038.stdout:(18/42): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 339 kB/s | 246 kB 00:00 2026-02-23T10:22:29.050 INFO:teuthology.orchestra.run.trial038.stdout:(19/42): lua-5.4.4-4.el9.x86_64.rpm 785 kB/s | 188 kB 00:00 2026-02-23T10:22:29.291 INFO:teuthology.orchestra.run.trial038.stdout:(20/42): python3-libstoragemgmt-1.10.1-1.el9.x8 735 kB/s | 177 kB 00:00 2026-02-23T10:22:29.453 INFO:teuthology.orchestra.run.trial038.stdout:(21/42): python3-packaging-20.9-5.el9.noarch.rp 477 kB/s | 77 kB 00:00 2026-02-23T10:22:29.552 INFO:teuthology.orchestra.run.trial038.stdout:(22/42): python3-toml-0.10.2-6.el9.noarch.rpm 417 kB/s | 42 kB 00:00 2026-02-23T10:22:29.592 INFO:teuthology.orchestra.run.trial038.stdout:(23/42): lua-devel-5.4.4-4.el9.x86_64.rpm 564 kB/s | 22 kB 00:00 2026-02-23T10:22:29.753 INFO:teuthology.orchestra.run.trial038.stdout:(24/42): luarocks-3.9.2-5.el9.noarch.rpm 941 kB/s | 151 kB 00:00 2026-02-23T10:22:29.801 INFO:teuthology.orchestra.run.trial038.stdout:(25/42): python3-autocommand-2.2.2-8.el9.noarch 609 kB/s | 29 kB 00:00 2026-02-23T10:22:29.831 INFO:teuthology.orchestra.run.trial038.stdout:(26/42): python3-backports-tarfile-1.2.0-1.el9. 1.9 MB/s | 60 kB 00:00 2026-02-23T10:22:29.860 INFO:teuthology.orchestra.run.trial038.stdout:(27/42): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.5 MB/s | 43 kB 00:00 2026-02-23T10:22:29.902 INFO:teuthology.orchestra.run.trial038.stdout:(28/42): python3-cheroot-10.0.1-4.el9.noarch.rp 4.0 MB/s | 173 kB 00:00 2026-02-23T10:22:29.956 INFO:teuthology.orchestra.run.trial038.stdout:(29/42): python3-cherrypy-18.6.1-2.el9.noarch.r 6.6 MB/s | 358 kB 00:00 2026-02-23T10:22:29.980 INFO:teuthology.orchestra.run.trial038.stdout:(30/42): python3-jaraco-8.2.1-3.el9.noarch.rpm 440 kB/s | 11 kB 00:00 2026-02-23T10:22:30.004 INFO:teuthology.orchestra.run.trial038.stdout:(31/42): python3-jaraco-classes-3.2.1-5.el9.noa 723 kB/s | 18 kB 00:00 2026-02-23T10:22:30.029 INFO:teuthology.orchestra.run.trial038.stdout:(32/42): python3-jaraco-collections-3.0.0-8.el9 950 kB/s | 23 kB 00:00 2026-02-23T10:22:30.053 INFO:teuthology.orchestra.run.trial038.stdout:(33/42): python3-jaraco-context-6.0.1-3.el9.noa 807 kB/s | 20 kB 00:00 2026-02-23T10:22:30.078 INFO:teuthology.orchestra.run.trial038.stdout:(34/42): python3-jaraco-functools-3.5.0-2.el9.n 795 kB/s | 19 kB 00:00 2026-02-23T10:22:30.102 INFO:teuthology.orchestra.run.trial038.stdout:(35/42): python3-jaraco-text-4.0.0-2.el9.noarch 1.1 MB/s | 26 kB 00:00 2026-02-23T10:22:30.128 INFO:teuthology.orchestra.run.trial038.stdout:(36/42): python3-more-itertools-8.12.0-2.el9.no 3.0 MB/s | 79 kB 00:00 2026-02-23T10:22:30.152 INFO:teuthology.orchestra.run.trial038.stdout:(37/42): python3-portend-3.1.0-2.el9.noarch.rpm 671 kB/s | 16 kB 00:00 2026-02-23T10:22:30.179 INFO:teuthology.orchestra.run.trial038.stdout:(38/42): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.3 MB/s | 90 kB 00:00 2026-02-23T10:22:30.204 INFO:teuthology.orchestra.run.trial038.stdout:(39/42): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2026-02-23T10:22:30.228 INFO:teuthology.orchestra.run.trial038.stdout:(40/42): python3-zc-lockfile-2.0-10.el9.noarch. 817 kB/s | 20 kB 00:00 2026-02-23T10:22:30.956 INFO:teuthology.orchestra.run.trial153.stdout:(12/12): ceph-test-20.3.0-5439.ge371433f.el9.x8 1.7 MB/s | 126 MB 01:15 2026-02-23T10:22:30.957 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:22:30.957 INFO:teuthology.orchestra.run.trial153.stdout:Total 1.7 MB/s | 131 MB 01:15 2026-02-23T10:22:31.026 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:22:31.038 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:22:31.038 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:22:31.466 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:22:31.466 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:22:31.918 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:22:31.927 INFO:teuthology.orchestra.run.trial153.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-23T10:22:31.936 INFO:teuthology.orchestra.run.trial153.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-23T10:22:31.941 INFO:teuthology.orchestra.run.trial153.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-23T10:22:31.945 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-23T10:22:31.948 INFO:teuthology.orchestra.run.trial153.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-23T10:22:31.952 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-23T10:22:31.953 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-23T10:22:31.970 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-23T10:22:31.973 INFO:teuthology.orchestra.run.trial153.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-23T10:22:31.983 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-23T10:22:31.991 INFO:teuthology.orchestra.run.trial153.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-23T10:22:31.992 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libcephsqlite-2:20.3.0-5439.ge371433f.el9.x86_64 11/12 2026-02-23T10:22:33.656 INFO:teuthology.orchestra.run.trial038.stdout:(41/42): ceph-osd-classic-20.3.0-5439.ge371433f 2.3 MB/s | 15 MB 00:06 2026-02-23T10:22:38.869 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5439.ge371433f.el9.x86_64 11/12 2026-02-23T10:22:38.886 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-test-2:20.3.0-5439.ge371433f.el9.x86_64 12/12 2026-02-23T10:22:39.647 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-test-2:20.3.0-5439.ge371433f.el9.x86_64 12/12 2026-02-23T10:22:39.647 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-test-2:20.3.0-5439.ge371433f.el9.x86_64 1/12 2026-02-23T10:22:39.647 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libcephsqlite-2:20.3.0-5439.ge371433f.el9.x86_64 2/12 2026-02-23T10:22:39.648 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-23T10:22:39.648 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-23T10:22:39.648 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-23T10:22:39.648 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-23T10:22:39.648 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-23T10:22:39.648 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-23T10:22:39.648 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-23T10:22:39.648 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-23T10:22:39.648 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-23T10:22:39.702 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-23T10:22:39.702 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:22:39.702 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:22:39.702 INFO:teuthology.orchestra.run.trial153.stdout: ceph-test-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:22:39.702 INFO:teuthology.orchestra.run.trial153.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-23T10:22:39.702 INFO:teuthology.orchestra.run.trial153.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-23T10:22:39.702 INFO:teuthology.orchestra.run.trial153.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-23T10:22:39.702 INFO:teuthology.orchestra.run.trial153.stdout: libcephsqlite-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:22:39.702 INFO:teuthology.orchestra.run.trial153.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-23T10:22:39.703 INFO:teuthology.orchestra.run.trial153.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-23T10:22:39.703 INFO:teuthology.orchestra.run.trial153.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-23T10:22:39.703 INFO:teuthology.orchestra.run.trial153.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-23T10:22:39.703 INFO:teuthology.orchestra.run.trial153.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-23T10:22:39.703 INFO:teuthology.orchestra.run.trial153.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-23T10:22:39.703 INFO:teuthology.orchestra.run.trial153.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-23T10:22:39.703 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:22:39.703 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:22:39.743 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph 2026-02-23T10:22:40.046 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:01:46 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:22:40.129 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:22:40.130 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:22:40.130 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repository Size 2026-02-23T10:22:40.130 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:22:40.130 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout: ceph x86_64 2:20.3.0-5439.ge371433f.el9 ceph 6.5 k 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout:Installing dependencies: 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mds x86_64 2:20.3.0-5439.ge371433f.el9 ceph 2.0 M 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr x86_64 2:20.3.0-5439.ge371433f.el9 ceph 937 k 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 298 k 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mon x86_64 2:20.3.0-5439.ge371433f.el9 ceph 4.9 M 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout: ceph-osd x86_64 2:20.3.0-5439.ge371433f.el9 ceph 164 k 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout: ceph-osd-classic x86_64 2:20.3.0-5439.ge371433f.el9 ceph 15 M 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout: ceph-osd-crimson x86_64 2:20.3.0-5439.ge371433f.el9 ceph 87 M 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-23T10:22:40.131 INFO:teuthology.orchestra.run.trial153.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-23T10:22:40.132 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco-collections 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: noarch 3.0.0-8.el9 epel 23 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-23T10:22:40.133 INFO:teuthology.orchestra.run.trial153.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-23T10:22:40.134 INFO:teuthology.orchestra.run.trial153.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-23T10:22:40.134 INFO:teuthology.orchestra.run.trial153.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-23T10:22:40.134 INFO:teuthology.orchestra.run.trial153.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-23T10:22:40.134 INFO:teuthology.orchestra.run.trial153.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-23T10:22:40.134 INFO:teuthology.orchestra.run.trial153.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-23T10:22:40.134 INFO:teuthology.orchestra.run.trial153.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-23T10:22:40.134 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:22:40.134 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:22:40.134 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:22:40.134 INFO:teuthology.orchestra.run.trial153.stdout:Install 42 Packages 2026-02-23T10:22:40.134 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:22:40.135 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 114 M 2026-02-23T10:22:40.135 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 961 M 2026-02-23T10:22:40.135 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:22:41.206 INFO:teuthology.orchestra.run.trial153.stdout:(1/42): ceph-20.3.0-5439.ge371433f.el9.x86_64.r 24 kB/s | 6.5 kB 00:00 2026-02-23T10:22:41.860 INFO:teuthology.orchestra.run.trial153.stdout:(2/42): ceph-mgr-20.3.0-5439.ge371433f.el9.x86_ 1.0 MB/s | 937 kB 00:00 2026-02-23T10:22:41.997 INFO:teuthology.orchestra.run.trial153.stdout:(3/42): ceph-osd-20.3.0-5439.ge371433f.el9.x86_ 1.2 MB/s | 164 kB 00:00 2026-02-23T10:22:42.368 INFO:teuthology.orchestra.run.trial153.stdout:(4/42): ceph-mds-20.3.0-5439.ge371433f.el9.x86_ 1.4 MB/s | 2.0 MB 00:01 2026-02-23T10:22:44.908 INFO:teuthology.orchestra.run.trial153.stdout:(5/42): ceph-mon-20.3.0-5439.ge371433f.el9.x86_ 1.3 MB/s | 4.9 MB 00:03 2026-02-23T10:22:45.183 INFO:teuthology.orchestra.run.trial153.stdout:(6/42): ceph-mgr-modules-core-20.3.0-5439.ge371 1.1 MB/s | 298 kB 00:00 2026-02-23T10:22:45.472 INFO:teuthology.orchestra.run.trial153.stdout:(7/42): ledmon-libs-1.1.0-3.el9.x86_64.rpm 140 kB/s | 40 kB 00:00 2026-02-23T10:22:45.664 INFO:teuthology.orchestra.run.trial153.stdout:(8/42): libconfig-1.7.2-9.el9.x86_64.rpm 375 kB/s | 72 kB 00:00 2026-02-23T10:22:45.896 INFO:teuthology.orchestra.run.trial153.stdout:(9/42): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2026-02-23T10:22:46.179 INFO:teuthology.orchestra.run.trial153.stdout:(10/42): python3-cryptography-36.0.1-5.el9.x86_ 4.4 MB/s | 1.2 MB 00:00 2026-02-23T10:22:46.261 INFO:teuthology.orchestra.run.trial153.stdout:(11/42): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2026-02-23T10:22:46.344 INFO:teuthology.orchestra.run.trial153.stdout:(12/42): python3-pycparser-2.20-6.el9.noarch.rp 1.6 MB/s | 135 kB 00:00 2026-02-23T10:22:46.428 INFO:teuthology.orchestra.run.trial153.stdout:(13/42): python3-pyparsing-2.4.7-9.el9.noarch.r 1.8 MB/s | 150 kB 00:00 2026-02-23T10:22:46.510 INFO:teuthology.orchestra.run.trial153.stdout:(14/42): python3-requests-2.25.1-10.el9.noarch. 1.5 MB/s | 126 kB 00:00 2026-02-23T10:22:46.597 INFO:teuthology.orchestra.run.trial153.stdout:(15/42): python3-urllib3-1.26.5-7.el9.noarch.rp 2.5 MB/s | 218 kB 00:00 2026-02-23T10:22:46.681 INFO:teuthology.orchestra.run.trial153.stdout:(16/42): unzip-6.0-59.el9.x86_64.rpm 2.1 MB/s | 182 kB 00:00 2026-02-23T10:22:46.768 INFO:teuthology.orchestra.run.trial153.stdout:(17/42): zip-3.0-35.el9.x86_64.rpm 3.0 MB/s | 266 kB 00:00 2026-02-23T10:22:47.536 INFO:teuthology.orchestra.run.trial153.stdout:(18/42): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 321 kB/s | 246 kB 00:00 2026-02-23T10:22:48.265 INFO:teuthology.orchestra.run.trial153.stdout:(19/42): lua-5.4.4-4.el9.x86_64.rpm 258 kB/s | 188 kB 00:00 2026-02-23T10:22:48.990 INFO:teuthology.orchestra.run.trial153.stdout:(20/42): python3-libstoragemgmt-1.10.1-1.el9.x8 244 kB/s | 177 kB 00:00 2026-02-23T10:22:49.622 INFO:teuthology.orchestra.run.trial153.stdout:(21/42): python3-packaging-20.9-5.el9.noarch.rp 122 kB/s | 77 kB 00:00 2026-02-23T10:22:50.134 INFO:teuthology.orchestra.run.trial153.stdout:(22/42): ceph-osd-classic-20.3.0-5439.ge371433f 1.8 MB/s | 15 MB 00:08 2026-02-23T10:22:50.196 INFO:teuthology.orchestra.run.trial153.stdout:(23/42): python3-toml-0.10.2-6.el9.noarch.rpm 72 kB/s | 42 kB 00:00 2026-02-23T10:22:50.541 INFO:teuthology.orchestra.run.trial153.stdout:(24/42): lua-devel-5.4.4-4.el9.x86_64.rpm 55 kB/s | 22 kB 00:00 2026-02-23T10:22:50.757 INFO:teuthology.orchestra.run.trial153.stdout:(25/42): luarocks-3.9.2-5.el9.noarch.rpm 270 kB/s | 151 kB 00:00 2026-02-23T10:22:50.836 INFO:teuthology.orchestra.run.trial153.stdout:(26/42): python3-backports-tarfile-1.2.0-1.el9. 756 kB/s | 60 kB 00:00 2026-02-23T10:22:50.927 INFO:teuthology.orchestra.run.trial153.stdout:(27/42): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 480 kB/s | 43 kB 00:00 2026-02-23T10:22:50.943 INFO:teuthology.orchestra.run.trial153.stdout:(28/42): python3-autocommand-2.2.2-8.el9.noarch 73 kB/s | 29 kB 00:00 2026-02-23T10:22:51.169 INFO:teuthology.orchestra.run.trial153.stdout:(29/42): python3-cheroot-10.0.1-4.el9.noarch.rp 718 kB/s | 173 kB 00:00 2026-02-23T10:22:51.255 INFO:teuthology.orchestra.run.trial153.stdout:(30/42): python3-jaraco-8.2.1-3.el9.noarch.rpm 124 kB/s | 11 kB 00:00 2026-02-23T10:22:51.340 INFO:teuthology.orchestra.run.trial153.stdout:(31/42): python3-jaraco-classes-3.2.1-5.el9.noa 208 kB/s | 18 kB 00:00 2026-02-23T10:22:51.417 INFO:teuthology.orchestra.run.trial153.stdout:(32/42): python3-cherrypy-18.6.1-2.el9.noarch.r 757 kB/s | 358 kB 00:00 2026-02-23T10:22:51.427 INFO:teuthology.orchestra.run.trial153.stdout:(33/42): python3-jaraco-collections-3.0.0-8.el9 266 kB/s | 23 kB 00:00 2026-02-23T10:22:51.506 INFO:teuthology.orchestra.run.trial153.stdout:(34/42): python3-jaraco-context-6.0.1-3.el9.noa 220 kB/s | 20 kB 00:00 2026-02-23T10:22:51.506 INFO:teuthology.orchestra.run.trial153.stdout:(35/42): python3-jaraco-functools-3.5.0-2.el9.n 245 kB/s | 19 kB 00:00 2026-02-23T10:22:51.593 INFO:teuthology.orchestra.run.trial153.stdout:(36/42): python3-jaraco-text-4.0.0-2.el9.noarch 305 kB/s | 26 kB 00:00 2026-02-23T10:22:51.665 INFO:teuthology.orchestra.run.trial153.stdout:(37/42): python3-more-itertools-8.12.0-2.el9.no 496 kB/s | 79 kB 00:00 2026-02-23T10:22:51.671 INFO:teuthology.orchestra.run.trial153.stdout:(38/42): python3-portend-3.1.0-2.el9.noarch.rpm 208 kB/s | 16 kB 00:00 2026-02-23T10:22:51.759 INFO:teuthology.orchestra.run.trial153.stdout:(39/42): python3-tempora-5.0.0-2.el9.noarch.rpm 408 kB/s | 36 kB 00:00 2026-02-23T10:22:51.822 INFO:teuthology.orchestra.run.trial153.stdout:(40/42): python3-pyOpenSSL-21.0.0-1.el9.noarch. 574 kB/s | 90 kB 00:00 2026-02-23T10:22:51.848 INFO:teuthology.orchestra.run.trial153.stdout:(41/42): python3-zc-lockfile-2.0-10.el9.noarch. 225 kB/s | 20 kB 00:00 2026-02-23T10:23:18.951 INFO:teuthology.orchestra.run.trial038.stdout:(42/42): ceph-osd-crimson-20.3.0-5439.ge371433f 1.7 MB/s | 87 MB 00:51 2026-02-23T10:23:18.955 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:23:18.955 INFO:teuthology.orchestra.run.trial038.stdout:Total 2.1 MB/s | 114 MB 00:53 2026-02-23T10:23:19.079 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:23:19.098 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:23:19.098 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:23:19.496 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:23:19.496 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:23:19.933 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:23:19.935 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/42 2026-02-23T10:23:19.940 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/42 2026-02-23T10:23:19.945 INFO:teuthology.orchestra.run.trial038.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/42 2026-02-23T10:23:19.947 INFO:teuthology.orchestra.run.trial038.stdout: Installing : unzip-6.0-59.el9.x86_64 4/42 2026-02-23T10:23:19.953 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/42 2026-02-23T10:23:19.976 INFO:teuthology.orchestra.run.trial038.stdout: Installing : zip-3.0-35.el9.x86_64 6/42 2026-02-23T10:23:19.980 INFO:teuthology.orchestra.run.trial038.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/42 2026-02-23T10:23:19.985 INFO:teuthology.orchestra.run.trial038.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/42 2026-02-23T10:23:19.989 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/42 2026-02-23T10:23:19.995 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/42 2026-02-23T10:23:19.999 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/42 2026-02-23T10:23:20.005 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/42 2026-02-23T10:23:20.010 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/42 2026-02-23T10:23:20.014 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/42 2026-02-23T10:23:20.019 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/42 2026-02-23T10:23:20.023 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/42 2026-02-23T10:23:20.030 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/42 2026-02-23T10:23:20.033 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/42 2026-02-23T10:23:20.056 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/42 2026-02-23T10:23:20.068 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/42 2026-02-23T10:23:20.073 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/42 2026-02-23T10:23:20.083 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/42 2026-02-23T10:23:20.089 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-23T10:23:20.103 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/42 2026-02-23T10:23:20.118 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/42 2026-02-23T10:23:20.193 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/42 2026-02-23T10:23:20.201 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/42 2026-02-23T10:23:20.224 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/42 2026-02-23T10:23:20.253 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/42 2026-02-23T10:23:20.259 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/42 2026-02-23T10:23:20.310 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/42 2026-02-23T10:23:20.351 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5439.ge371433f.el9. 32/42 2026-02-23T10:23:20.352 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 33/42 2026-02-23T10:23:20.375 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 33/42 2026-02-23T10:23:20.375 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:23:20.375 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-23T10:23:20.375 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-23T10:23:20.375 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-23T10:23:20.375 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:20.380 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/42 2026-02-23T10:23:20.396 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-23T10:23:20.396 INFO:teuthology.orchestra.run.trial038.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-23T10:23:20.396 INFO:teuthology.orchestra.run.trial038.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-23T10:23:20.396 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:20.403 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-23T10:23:20.426 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-23T10:23:20.426 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-23T10:23:20.426 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:21.039 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/42 2026-02-23T10:23:26.465 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_6 37/42 2026-02-23T10:23:26.475 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-osd-crimson-2:20.3.0-5439.ge371433f.el9.x86_6 38/42 2026-02-23T10:23:26.476 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 39/42 2026-02-23T10:23:26.500 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 39/42 2026-02-23T10:23:26.500 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:23:26.500 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-23T10:23:26.500 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-23T10:23:26.501 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-23T10:23:26.501 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:26.709 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mon-2:20.3.0-5439.ge371433f.el9.x86_64 40/42 2026-02-23T10:23:26.729 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mon-2:20.3.0-5439.ge371433f.el9.x86_64 40/42 2026-02-23T10:23:26.730 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:23:26.730 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-23T10:23:26.730 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-23T10:23:26.730 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-23T10:23:26.730 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:26.813 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mds-2:20.3.0-5439.ge371433f.el9.x86_64 41/42 2026-02-23T10:23:26.833 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mds-2:20.3.0-5439.ge371433f.el9.x86_64 41/42 2026-02-23T10:23:26.833 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:23:26.833 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-23T10:23:26.833 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-23T10:23:26.834 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-23T10:23:26.834 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:26.834 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-2:20.3.0-5439.ge371433f.el9.x86_64 42/42 2026-02-23T10:23:26.840 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_6 42/42 2026-02-23T10:23:26.862 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5439.ge371433f.el9.x86_6 42/42 2026-02-23T10:23:27.824 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-2:20.3.0-5439.ge371433f.el9.x86_64 42/42 2026-02-23T10:23:27.824 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-2:20.3.0-5439.ge371433f.el9.x86_64 1/42 2026-02-23T10:23:27.824 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mds-2:20.3.0-5439.ge371433f.el9.x86_64 2/42 2026-02-23T10:23:27.824 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 3/42 2026-02-23T10:23:27.824 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mon-2:20.3.0-5439.ge371433f.el9.x86_64 4/42 2026-02-23T10:23:27.824 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 5/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_6 6/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5439.ge371433f.el9.x86_6 7/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5439.ge371433f.el9. 8/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 9/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 10/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 12/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 15/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 16/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 17/42 2026-02-23T10:23:27.825 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : unzip-6.0-59.el9.x86_64 18/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : zip-3.0-35.el9.x86_64 19/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 20/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : lua-5.4.4-4.el9.x86_64 21/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 22/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/42 2026-02-23T10:23:27.826 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/42 2026-02-23T10:23:27.827 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/42 2026-02-23T10:23:27.827 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/42 2026-02-23T10:23:27.827 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/42 2026-02-23T10:23:27.827 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/42 2026-02-23T10:23:27.827 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/42 2026-02-23T10:23:27.827 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/42 2026-02-23T10:23:27.827 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/42 2026-02-23T10:23:27.827 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 38/42 2026-02-23T10:23:27.827 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 39/42 2026-02-23T10:23:27.827 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 40/42 2026-02-23T10:23:27.827 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 41/42 2026-02-23T10:23:27.881 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 42/42 2026-02-23T10:23:27.881 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:27.881 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: ceph-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mds-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-modules-core-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mon-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-crimson-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-23T10:23:27.882 INFO:teuthology.orchestra.run.trial038.stdout: lua-5.4.4-4.el9.x86_64 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-23T10:23:27.883 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-ply-3.11-14.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-23T10:23:27.884 INFO:teuthology.orchestra.run.trial038.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-23T10:23:27.885 INFO:teuthology.orchestra.run.trial038.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-23T10:23:27.885 INFO:teuthology.orchestra.run.trial038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-23T10:23:27.885 INFO:teuthology.orchestra.run.trial038.stdout: unzip-6.0-59.el9.x86_64 2026-02-23T10:23:27.885 INFO:teuthology.orchestra.run.trial038.stdout: zip-3.0-35.el9.x86_64 2026-02-23T10:23:27.885 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:27.885 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:23:27.924 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-base 2026-02-23T10:23:28.240 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:32 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:23:28.304 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:23:28.322 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:23:28.322 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:23:28.322 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:23:28.344 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install cephadm 2026-02-23T10:23:28.650 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:32 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:23:28.731 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:23:28.731 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:23:28.732 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-23T10:23:28.732 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:23:28.732 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:23:28.732 INFO:teuthology.orchestra.run.trial038.stdout: cephadm noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 1.0 M 2026-02-23T10:23:28.732 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:28.732 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:23:28.732 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:23:28.732 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-23T10:23:28.732 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:28.732 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 1.0 M 2026-02-23T10:23:28.733 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 1.0 M 2026-02-23T10:23:28.733 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:23:29.737 INFO:teuthology.orchestra.run.trial038.stdout:cephadm-20.3.0-5439.ge371433f.el9.noarch.rpm 1.0 MB/s | 1.0 MB 00:01 2026-02-23T10:23:29.738 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:23:29.738 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.0 MB/s | 1.0 MB 00:01 2026-02-23T10:23:29.738 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:23:29.741 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:23:29.741 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:23:29.745 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:23:29.745 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:23:29.763 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:23:29.825 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: cephadm-2:20.3.0-5439.ge371433f.el9.noarch 1/1 2026-02-23T10:23:29.839 INFO:teuthology.orchestra.run.trial038.stdout: Installing : cephadm-2:20.3.0-5439.ge371433f.el9.noarch 1/1 2026-02-23T10:23:29.904 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: cephadm-2:20.3.0-5439.ge371433f.el9.noarch 1/1 2026-02-23T10:23:29.959 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : cephadm-2:20.3.0-5439.ge371433f.el9.noarch 1/1 2026-02-23T10:23:29.960 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:29.960 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:23:29.960 INFO:teuthology.orchestra.run.trial038.stdout: cephadm-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:23:29.960 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:29.960 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:23:29.987 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-immutable-object-cache 2026-02-23T10:23:30.295 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:34 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:23:30.376 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:23:30.377 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:23:30.377 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-23T10:23:30.377 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:23:30.378 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:23:30.378 INFO:teuthology.orchestra.run.trial038.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5439.ge371433f.el9 ceph 149 k 2026-02-23T10:23:30.378 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:30.378 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:23:30.378 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:23:30.378 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-23T10:23:30.378 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:30.378 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 149 k 2026-02-23T10:23:30.378 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 455 k 2026-02-23T10:23:30.378 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:23:30.824 INFO:teuthology.orchestra.run.trial038.stdout:ceph-immutable-object-cache-20.3.0-5439.ge37143 334 kB/s | 149 kB 00:00 2026-02-23T10:23:30.824 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:23:30.824 INFO:teuthology.orchestra.run.trial038.stdout:Total 333 kB/s | 149 kB 00:00 2026-02-23T10:23:30.825 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:23:30.829 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:23:30.829 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:23:30.855 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:23:30.856 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:23:30.905 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:23:30.907 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5439.ge371433f. 1/1 2026-02-23T10:23:30.929 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5439.ge371433f. 1/1 2026-02-23T10:23:30.929 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:23:30.929 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-23T10:23:30.929 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:31.210 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5439.ge371433f. 1/1 2026-02-23T10:23:31.210 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:31.210 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:23:31.210 INFO:teuthology.orchestra.run.trial038.stdout: ceph-immutable-object-cache-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:31.211 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:31.211 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:23:31.240 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr 2026-02-23T10:23:31.551 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:35 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:23:31.614 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:23:31.631 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:23:31.632 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:23:31.632 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:23:31.653 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-dashboard 2026-02-23T10:23:31.953 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:35 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:23:32.035 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:23:32.035 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:23:32.036 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-23T10:23:32.036 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:23:32.036 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:23:32.036 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 51 M 2026-02-23T10:23:32.036 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-23T10:23:32.036 INFO:teuthology.orchestra.run.trial038.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-23T10:23:32.036 INFO:teuthology.orchestra.run.trial038.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 57 k 2026-02-23T10:23:32.036 INFO:teuthology.orchestra.run.trial038.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 17 k 2026-02-23T10:23:32.036 INFO:teuthology.orchestra.run.trial038.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-23T10:23:32.036 INFO:teuthology.orchestra.run.trial038.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-23T10:23:32.036 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-23T10:23:32.037 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-23T10:23:32.037 INFO:teuthology.orchestra.run.trial038.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-23T10:23:32.037 INFO:teuthology.orchestra.run.trial038.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-23T10:23:32.037 INFO:teuthology.orchestra.run.trial038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-23T10:23:32.037 INFO:teuthology.orchestra.run.trial038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-23T10:23:32.037 INFO:teuthology.orchestra.run.trial038.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-23T10:23:32.037 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:32.037 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:23:32.037 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:23:32.037 INFO:teuthology.orchestra.run.trial038.stdout:Install 13 Packages 2026-02-23T10:23:32.037 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:32.038 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 55 M 2026-02-23T10:23:32.038 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 262 M 2026-02-23T10:23:32.038 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:23:32.839 INFO:teuthology.orchestra.run.trial038.stdout:(1/13): ceph-prometheus-alerts-20.3.0-5439.ge37 52 kB/s | 17 kB 00:00 2026-02-23T10:23:32.854 INFO:teuthology.orchestra.run.trial038.stdout:(2/13): python3-jmespath-1.0.1-1.el9.noarch.rpm 3.2 MB/s | 48 kB 00:00 2026-02-23T10:23:32.867 INFO:teuthology.orchestra.run.trial038.stdout:(3/13): python3-protobuf-3.14.0-17.el9.noarch.r 20 MB/s | 267 kB 00:00 2026-02-23T10:23:32.918 INFO:teuthology.orchestra.run.trial038.stdout:(4/13): ceph-grafana-dashboards-20.3.0-5439.ge3 139 kB/s | 57 kB 00:00 2026-02-23T10:23:33.060 INFO:teuthology.orchestra.run.trial038.stdout:(5/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 3.8 MB/s | 551 kB 00:00 2026-02-23T10:23:33.065 INFO:teuthology.orchestra.run.trial038.stdout:(6/13): protobuf-compiler-3.14.0-17.el9.x86_64. 4.2 MB/s | 862 kB 00:00 2026-02-23T10:23:33.082 INFO:teuthology.orchestra.run.trial038.stdout:(7/13): grpc-data-1.46.7-10.el9.noarch.rpm 918 kB/s | 19 kB 00:00 2026-02-23T10:23:33.107 INFO:teuthology.orchestra.run.trial038.stdout:(8/13): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.6 MB/s | 144 kB 00:00 2026-02-23T10:23:33.128 INFO:teuthology.orchestra.run.trial038.stdout:(9/13): python3-repoze-lru-0.7-16.el9.noarch.rp 1.4 MB/s | 31 kB 00:00 2026-02-23T10:23:33.153 INFO:teuthology.orchestra.run.trial038.stdout:(10/13): python3-routes-2.5.1-5.el9.noarch.rpm 7.6 MB/s | 188 kB 00:00 2026-02-23T10:23:33.175 INFO:teuthology.orchestra.run.trial038.stdout:(11/13): python3-xmltodict-0.12.0-15.el9.noarch 973 kB/s | 22 kB 00:00 2026-02-23T10:23:33.270 INFO:teuthology.orchestra.run.trial038.stdout:(12/13): python3-grpcio-1.46.7-10.el9.x86_64.rp 9.9 MB/s | 2.0 MB 00:00 2026-02-23T10:23:42.666 INFO:teuthology.orchestra.run.trial153.stdout:(42/42): ceph-osd-crimson-20.3.0-5439.ge371433f 1.4 MB/s | 87 MB 01:00 2026-02-23T10:23:42.669 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:23:42.670 INFO:teuthology.orchestra.run.trial153.stdout:Total 1.8 MB/s | 114 MB 01:02 2026-02-23T10:23:42.787 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:23:42.808 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:23:42.808 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:23:43.201 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:23:43.201 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:23:43.636 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:23:43.638 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/42 2026-02-23T10:23:43.642 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/42 2026-02-23T10:23:43.647 INFO:teuthology.orchestra.run.trial153.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/42 2026-02-23T10:23:43.649 INFO:teuthology.orchestra.run.trial153.stdout: Installing : unzip-6.0-59.el9.x86_64 4/42 2026-02-23T10:23:43.655 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/42 2026-02-23T10:23:43.679 INFO:teuthology.orchestra.run.trial153.stdout: Installing : zip-3.0-35.el9.x86_64 6/42 2026-02-23T10:23:43.684 INFO:teuthology.orchestra.run.trial153.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/42 2026-02-23T10:23:43.689 INFO:teuthology.orchestra.run.trial153.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/42 2026-02-23T10:23:43.694 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/42 2026-02-23T10:23:43.700 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/42 2026-02-23T10:23:43.704 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/42 2026-02-23T10:23:43.710 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/42 2026-02-23T10:23:43.715 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/42 2026-02-23T10:23:43.719 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/42 2026-02-23T10:23:43.724 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/42 2026-02-23T10:23:43.728 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/42 2026-02-23T10:23:43.735 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/42 2026-02-23T10:23:43.738 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/42 2026-02-23T10:23:43.761 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/42 2026-02-23T10:23:43.774 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/42 2026-02-23T10:23:43.779 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/42 2026-02-23T10:23:43.788 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/42 2026-02-23T10:23:43.795 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-23T10:23:43.808 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/42 2026-02-23T10:23:43.824 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/42 2026-02-23T10:23:43.899 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/42 2026-02-23T10:23:43.907 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/42 2026-02-23T10:23:43.931 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/42 2026-02-23T10:23:43.961 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/42 2026-02-23T10:23:43.968 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/42 2026-02-23T10:23:44.020 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/42 2026-02-23T10:23:44.060 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5439.ge371433f.el9. 32/42 2026-02-23T10:23:44.061 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 33/42 2026-02-23T10:23:44.083 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 33/42 2026-02-23T10:23:44.084 INFO:teuthology.orchestra.run.trial153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:23:44.084 INFO:teuthology.orchestra.run.trial153.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-23T10:23:44.084 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-23T10:23:44.084 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-23T10:23:44.084 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:44.089 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/42 2026-02-23T10:23:44.104 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-23T10:23:44.104 INFO:teuthology.orchestra.run.trial153.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-23T10:23:44.104 INFO:teuthology.orchestra.run.trial153.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-23T10:23:44.104 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:44.112 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-23T10:23:44.134 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-23T10:23:44.134 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-23T10:23:44.134 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:44.738 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/42 2026-02-23T10:23:50.084 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_6 37/42 2026-02-23T10:23:50.094 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-osd-crimson-2:20.3.0-5439.ge371433f.el9.x86_6 38/42 2026-02-23T10:23:50.095 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 39/42 2026-02-23T10:23:50.120 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 39/42 2026-02-23T10:23:50.120 INFO:teuthology.orchestra.run.trial153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:23:50.120 INFO:teuthology.orchestra.run.trial153.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-23T10:23:50.120 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-23T10:23:50.121 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-23T10:23:50.121 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:50.326 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-mon-2:20.3.0-5439.ge371433f.el9.x86_64 40/42 2026-02-23T10:23:50.346 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-mon-2:20.3.0-5439.ge371433f.el9.x86_64 40/42 2026-02-23T10:23:50.346 INFO:teuthology.orchestra.run.trial153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:23:50.346 INFO:teuthology.orchestra.run.trial153.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-23T10:23:50.346 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-23T10:23:50.346 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-23T10:23:50.346 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:50.429 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-mds-2:20.3.0-5439.ge371433f.el9.x86_64 41/42 2026-02-23T10:23:50.448 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-mds-2:20.3.0-5439.ge371433f.el9.x86_64 41/42 2026-02-23T10:23:50.449 INFO:teuthology.orchestra.run.trial153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:23:50.449 INFO:teuthology.orchestra.run.trial153.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-23T10:23:50.449 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-23T10:23:50.449 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-23T10:23:50.449 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:50.450 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-2:20.3.0-5439.ge371433f.el9.x86_64 42/42 2026-02-23T10:23:50.456 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_6 42/42 2026-02-23T10:23:50.478 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5439.ge371433f.el9.x86_6 42/42 2026-02-23T10:23:51.433 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-2:20.3.0-5439.ge371433f.el9.x86_64 42/42 2026-02-23T10:23:51.434 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-2:20.3.0-5439.ge371433f.el9.x86_64 1/42 2026-02-23T10:23:51.434 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-mds-2:20.3.0-5439.ge371433f.el9.x86_64 2/42 2026-02-23T10:23:51.434 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 3/42 2026-02-23T10:23:51.434 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-mon-2:20.3.0-5439.ge371433f.el9.x86_64 4/42 2026-02-23T10:23:51.434 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 5/42 2026-02-23T10:23:51.434 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_6 6/42 2026-02-23T10:23:51.434 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5439.ge371433f.el9.x86_6 7/42 2026-02-23T10:23:51.434 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5439.ge371433f.el9. 8/42 2026-02-23T10:23:51.434 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 9/42 2026-02-23T10:23:51.434 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 10/42 2026-02-23T10:23:51.434 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 12/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 15/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 16/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 17/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : unzip-6.0-59.el9.x86_64 18/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : zip-3.0-35.el9.x86_64 19/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 20/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : lua-5.4.4-4.el9.x86_64 21/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 22/42 2026-02-23T10:23:51.435 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/42 2026-02-23T10:23:51.436 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/42 2026-02-23T10:23:51.437 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 38/42 2026-02-23T10:23:51.437 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 39/42 2026-02-23T10:23:51.437 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 40/42 2026-02-23T10:23:51.437 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 41/42 2026-02-23T10:23:51.491 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 42/42 2026-02-23T10:23:51.491 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:51.491 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:23:51.491 INFO:teuthology.orchestra.run.trial153.stdout: ceph-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:51.491 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mds-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr-modules-core-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mon-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: ceph-osd-crimson-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: lua-5.4.4-4.el9.x86_64 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-23T10:23:51.492 INFO:teuthology.orchestra.run.trial153.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-23T10:23:51.493 INFO:teuthology.orchestra.run.trial153.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-ply-3.11-14.el9.noarch 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-23T10:23:51.494 INFO:teuthology.orchestra.run.trial153.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-23T10:23:51.495 INFO:teuthology.orchestra.run.trial153.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-23T10:23:51.495 INFO:teuthology.orchestra.run.trial153.stdout: unzip-6.0-59.el9.x86_64 2026-02-23T10:23:51.495 INFO:teuthology.orchestra.run.trial153.stdout: zip-3.0-35.el9.x86_64 2026-02-23T10:23:51.495 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:51.495 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:23:51.534 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-base 2026-02-23T10:23:51.848 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:02:57 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:23:51.911 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:23:51.929 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:23:51.929 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:23:51.929 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:23:51.951 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install cephadm 2026-02-23T10:23:52.256 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:02:58 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:23:52.338 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:23:52.339 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:23:52.339 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repository Size 2026-02-23T10:23:52.339 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:23:52.339 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:23:52.339 INFO:teuthology.orchestra.run.trial153.stdout: cephadm noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 1.0 M 2026-02-23T10:23:52.339 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:52.339 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:23:52.340 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:23:52.340 INFO:teuthology.orchestra.run.trial153.stdout:Install 1 Package 2026-02-23T10:23:52.340 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:52.340 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 1.0 M 2026-02-23T10:23:52.340 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 1.0 M 2026-02-23T10:23:52.340 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:23:53.604 INFO:teuthology.orchestra.run.trial153.stdout:cephadm-20.3.0-5439.ge371433f.el9.noarch.rpm 824 kB/s | 1.0 MB 00:01 2026-02-23T10:23:53.606 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:23:53.606 INFO:teuthology.orchestra.run.trial153.stdout:Total 822 kB/s | 1.0 MB 00:01 2026-02-23T10:23:53.607 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:23:53.610 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:23:53.610 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:23:53.614 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:23:53.614 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:23:53.632 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:23:53.691 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: cephadm-2:20.3.0-5439.ge371433f.el9.noarch 1/1 2026-02-23T10:23:53.705 INFO:teuthology.orchestra.run.trial153.stdout: Installing : cephadm-2:20.3.0-5439.ge371433f.el9.noarch 1/1 2026-02-23T10:23:53.769 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: cephadm-2:20.3.0-5439.ge371433f.el9.noarch 1/1 2026-02-23T10:23:53.824 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : cephadm-2:20.3.0-5439.ge371433f.el9.noarch 1/1 2026-02-23T10:23:53.825 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:53.825 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:23:53.825 INFO:teuthology.orchestra.run.trial153.stdout: cephadm-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:23:53.825 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:53.825 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:23:53.850 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-immutable-object-cache 2026-02-23T10:23:54.162 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:00 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:23:54.244 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:23:54.244 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:23:54.245 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repo Size 2026-02-23T10:23:54.245 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:23:54.245 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:23:54.245 INFO:teuthology.orchestra.run.trial153.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5439.ge371433f.el9 ceph 149 k 2026-02-23T10:23:54.245 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:54.245 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:23:54.245 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:23:54.245 INFO:teuthology.orchestra.run.trial153.stdout:Install 1 Package 2026-02-23T10:23:54.245 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:54.245 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 149 k 2026-02-23T10:23:54.246 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 455 k 2026-02-23T10:23:54.246 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:23:54.728 INFO:teuthology.orchestra.run.trial153.stdout:ceph-immutable-object-cache-20.3.0-5439.ge37143 308 kB/s | 149 kB 00:00 2026-02-23T10:23:54.728 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:23:54.728 INFO:teuthology.orchestra.run.trial153.stdout:Total 308 kB/s | 149 kB 00:00 2026-02-23T10:23:54.729 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:23:54.733 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:23:54.733 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:23:54.759 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:23:54.760 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:23:54.809 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:23:54.810 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5439.ge371433f. 1/1 2026-02-23T10:23:54.833 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5439.ge371433f. 1/1 2026-02-23T10:23:54.833 INFO:teuthology.orchestra.run.trial153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:23:54.833 INFO:teuthology.orchestra.run.trial153.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-23T10:23:54.833 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:55.101 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5439.ge371433f. 1/1 2026-02-23T10:23:55.102 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:55.102 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:23:55.102 INFO:teuthology.orchestra.run.trial153.stdout: ceph-immutable-object-cache-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:23:55.102 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:55.102 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:23:55.128 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-mgr 2026-02-23T10:23:55.198 INFO:teuthology.orchestra.run.trial038.stdout:(13/13): ceph-mgr-dashboard-20.3.0-5439.ge37143 2.2 MB/s | 51 MB 00:22 2026-02-23T10:23:55.200 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:23:55.200 INFO:teuthology.orchestra.run.trial038.stdout:Total 2.4 MB/s | 55 MB 00:23 2026-02-23T10:23:55.226 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:23:55.236 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:23:55.236 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:23:55.440 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:01 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:23:55.447 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:23:55.448 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:23:55.503 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:23:55.521 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:23:55.521 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:23:55.522 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:23:55.543 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-mgr-dashboard 2026-02-23T10:23:55.676 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:23:55.682 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 1/13 2026-02-23T10:23:55.693 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 2/13 2026-02-23T10:23:55.695 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 3/13 2026-02-23T10:23:55.723 INFO:teuthology.orchestra.run.trial038.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 4/13 2026-02-23T10:23:55.766 INFO:teuthology.orchestra.run.trial038.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 5/13 2026-02-23T10:23:55.774 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 6/13 2026-02-23T10:23:55.802 INFO:teuthology.orchestra.run.trial038.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 7/13 2026-02-23T10:23:55.812 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 8/13 2026-02-23T10:23:55.819 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/13 2026-02-23T10:23:55.822 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 10/13 2026-02-23T10:23:55.831 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5439.ge371433f.el9 11/13 2026-02-23T10:23:55.845 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:01 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:23:55.928 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:23:55.928 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:23:55.928 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repository Size 2026-02-23T10:23:55.928 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 51 M 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout:Installing dependencies: 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 57 k 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 17 k 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-23T10:23:55.929 INFO:teuthology.orchestra.run.trial153.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-23T10:23:55.930 INFO:teuthology.orchestra.run.trial153.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-23T10:23:55.930 INFO:teuthology.orchestra.run.trial153.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-23T10:23:55.930 INFO:teuthology.orchestra.run.trial153.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-23T10:23:55.930 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:55.930 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:23:55.930 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:23:55.930 INFO:teuthology.orchestra.run.trial153.stdout:Install 13 Packages 2026-02-23T10:23:55.930 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:23:55.931 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 55 M 2026-02-23T10:23:55.931 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 262 M 2026-02-23T10:23:55.931 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:23:56.531 INFO:teuthology.orchestra.run.trial153.stdout:(1/13): ceph-prometheus-alerts-20.3.0-5439.ge37 51 kB/s | 17 kB 00:00 2026-02-23T10:23:56.598 INFO:teuthology.orchestra.run.trial153.stdout:(2/13): ceph-grafana-dashboards-20.3.0-5439.ge3 140 kB/s | 57 kB 00:00 2026-02-23T10:23:56.855 INFO:teuthology.orchestra.run.trial153.stdout:(3/13): python3-jmespath-1.0.1-1.el9.noarch.rpm 147 kB/s | 48 kB 00:00 2026-02-23T10:23:57.106 INFO:teuthology.orchestra.run.trial153.stdout:(4/13): python3-protobuf-3.14.0-17.el9.noarch.r 527 kB/s | 267 kB 00:00 2026-02-23T10:23:57.594 INFO:teuthology.orchestra.run.trial153.stdout:(5/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.1 MB/s | 551 kB 00:00 2026-02-23T10:23:57.663 INFO:teuthology.orchestra.run.trial153.stdout:(6/13): grpc-data-1.46.7-10.el9.noarch.rpm 280 kB/s | 19 kB 00:00 2026-02-23T10:23:57.877 INFO:teuthology.orchestra.run.trial153.stdout:(7/13): python3-grpcio-1.46.7-10.el9.x86_64.rpm 9.6 MB/s | 2.0 MB 00:00 2026-02-23T10:23:57.949 INFO:teuthology.orchestra.run.trial153.stdout:(8/13): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.0 MB/s | 144 kB 00:00 2026-02-23T10:23:58.019 INFO:teuthology.orchestra.run.trial153.stdout:(9/13): python3-repoze-lru-0.7-16.el9.noarch.rp 439 kB/s | 31 kB 00:00 2026-02-23T10:23:58.093 INFO:teuthology.orchestra.run.trial153.stdout:(10/13): python3-routes-2.5.1-5.el9.noarch.rpm 2.5 MB/s | 188 kB 00:00 2026-02-23T10:23:58.163 INFO:teuthology.orchestra.run.trial153.stdout:(11/13): python3-xmltodict-0.12.0-15.el9.noarch 317 kB/s | 22 kB 00:00 2026-02-23T10:23:58.426 INFO:teuthology.orchestra.run.trial153.stdout:(12/13): protobuf-compiler-3.14.0-17.el9.x86_64 548 kB/s | 862 kB 00:01 2026-02-23T10:23:58.727 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5439.ge371433f.el 12/13 2026-02-23T10:23:58.734 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5439.ge371433f.el9.noa 13/13 2026-02-23T10:23:59.022 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5439.ge371433f.el9.noa 13/13 2026-02-23T10:23:59.023 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5439.ge371433f.el 1/13 2026-02-23T10:23:59.023 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5439.ge371433f.el9.noa 2/13 2026-02-23T10:23:59.023 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5439.ge371433f.el9 3/13 2026-02-23T10:23:59.023 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 4/13 2026-02-23T10:23:59.023 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-23T10:23:59.023 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-23T10:23:59.023 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-23T10:23:59.023 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-23T10:23:59.023 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-23T10:23:59.024 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-23T10:23:59.024 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-23T10:23:59.024 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-23T10:23:59.079 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-23T10:23:59.079 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:59.079 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:23:59.079 INFO:teuthology.orchestra.run.trial038.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-23T10:23:59.079 INFO:teuthology.orchestra.run.trial038.stdout: ceph-grafana-dashboards-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-dashboard-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: ceph-prometheus-alerts-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-23T10:23:59.080 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:59.081 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:23:59.117 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-23T10:23:59.427 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:03 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:23:59.510 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:23:59.511 INFO:teuthology.orchestra.run.trial038.stdout:======================================================================================= 2026-02-23T10:23:59.512 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-23T10:23:59.512 INFO:teuthology.orchestra.run.trial038.stdout:======================================================================================= 2026-02-23T10:23:59.512 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:23:59.512 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 7.4 M 2026-02-23T10:23:59.512 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-23T10:23:59.512 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-23T10:23:59.512 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-23T10:23:59.513 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-23T10:23:59.513 INFO:teuthology.orchestra.run.trial038.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-23T10:23:59.513 INFO:teuthology.orchestra.run.trial038.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-23T10:23:59.513 INFO:teuthology.orchestra.run.trial038.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-23T10:23:59.513 INFO:teuthology.orchestra.run.trial038.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-23T10:23:59.513 INFO:teuthology.orchestra.run.trial038.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-23T10:23:59.513 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-23T10:23:59.513 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-23T10:23:59.513 INFO:teuthology.orchestra.run.trial038.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-23T10:23:59.514 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:59.514 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:23:59.514 INFO:teuthology.orchestra.run.trial038.stdout:======================================================================================= 2026-02-23T10:23:59.514 INFO:teuthology.orchestra.run.trial038.stdout:Install 12 Packages 2026-02-23T10:23:59.514 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:23:59.514 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 43 M 2026-02-23T10:23:59.514 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 234 M 2026-02-23T10:23:59.515 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:23:59.920 INFO:teuthology.orchestra.run.trial038.stdout:(1/12): libquadmath-11.5.0-14.el9.x86_64.rpm 3.8 MB/s | 184 kB 00:00 2026-02-23T10:23:59.937 INFO:teuthology.orchestra.run.trial038.stdout:(2/12): libgfortran-11.5.0-14.el9.x86_64.rpm 12 MB/s | 794 kB 00:00 2026-02-23T10:24:00.027 INFO:teuthology.orchestra.run.trial038.stdout:(3/12): flexiblas-3.0.4-9.el9.x86_64.rpm 277 kB/s | 30 kB 00:00 2026-02-23T10:24:00.129 INFO:teuthology.orchestra.run.trial038.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-9.el9.x 145 kB/s | 15 kB 00:00 2026-02-23T10:24:00.271 INFO:teuthology.orchestra.run.trial038.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 298 kB/s | 42 kB 00:00 2026-02-23T10:24:00.359 INFO:teuthology.orchestra.run.trial038.stdout:(6/12): flexiblas-netlib-3.0.4-9.el9.x86_64.rpm 7.1 MB/s | 3.0 MB 00:00 2026-02-23T10:24:00.548 INFO:teuthology.orchestra.run.trial038.stdout:(7/12): python3-devel-3.9.25-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2026-02-23T10:24:00.558 INFO:teuthology.orchestra.run.trial038.stdout:(8/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 18 MB/s | 5.3 MB 00:00 2026-02-23T10:24:00.730 INFO:teuthology.orchestra.run.trial038.stdout:(9/12): python3-numpy-f2py-1.23.5-2.el9.x86_64. 2.5 MB/s | 442 kB 00:00 2026-02-23T10:24:00.780 INFO:teuthology.orchestra.run.trial038.stdout:(10/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 26 MB/s | 6.1 MB 00:00 2026-02-23T10:24:01.301 INFO:teuthology.orchestra.run.trial038.stdout:(11/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 34 MB/s | 19 MB 00:00 2026-02-23T10:24:02.330 INFO:teuthology.orchestra.run.trial038.stdout:(12/12): ceph-mgr-diskprediction-local-20.3.0-5 3.0 MB/s | 7.4 MB 00:02 2026-02-23T10:24:02.331 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:02.331 INFO:teuthology.orchestra.run.trial038.stdout:Total 15 MB/s | 43 MB 00:02 2026-02-23T10:24:02.499 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:24:02.513 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:24:02.513 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:24:02.716 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:24:02.716 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:24:02.934 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:24:02.936 INFO:teuthology.orchestra.run.trial038.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 1/12 2026-02-23T10:24:02.943 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-23T10:24:02.976 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-23T10:24:02.981 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 4/12 2026-02-23T10:24:03.041 INFO:teuthology.orchestra.run.trial038.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-23T10:24:03.044 INFO:teuthology.orchestra.run.trial038.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-23T10:24:03.065 INFO:teuthology.orchestra.run.trial038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 7/12 2026-02-23T10:24:03.385 INFO:teuthology.orchestra.run.trial038.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 8/12 2026-02-23T10:24:03.457 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-23T10:24:04.116 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-23T10:24:04.593 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-23T10:24:04.594 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5439.ge3714 12/12 2026-02-23T10:24:04.862 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5439.ge3714 12/12 2026-02-23T10:24:04.862 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5439.ge3714 1/12 2026-02-23T10:24:04.862 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-23T10:24:04.862 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-23T10:24:04.863 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 4/12 2026-02-23T10:24:04.863 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 5/12 2026-02-23T10:24:04.863 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 6/12 2026-02-23T10:24:04.863 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-23T10:24:04.863 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-23T10:24:04.863 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 9/12 2026-02-23T10:24:04.863 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-23T10:24:04.863 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-23T10:24:04.921 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-23T10:24:04.921 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:04.921 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:24:04.921 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:24:04.921 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-23T10:24:04.921 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-23T10:24:04.921 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-23T10:24:04.921 INFO:teuthology.orchestra.run.trial038.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-23T10:24:04.922 INFO:teuthology.orchestra.run.trial038.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-23T10:24:04.922 INFO:teuthology.orchestra.run.trial038.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-23T10:24:04.922 INFO:teuthology.orchestra.run.trial038.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-23T10:24:04.922 INFO:teuthology.orchestra.run.trial038.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-23T10:24:04.922 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-23T10:24:04.922 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-23T10:24:04.922 INFO:teuthology.orchestra.run.trial038.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-23T10:24:04.922 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:04.922 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:04.960 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-rook 2026-02-23T10:24:05.268 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:09 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:05.351 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:05.352 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:05.352 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-23T10:24:05.353 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:05.353 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:24:05.353 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-rook noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 50 k 2026-02-23T10:24:05.353 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-23T10:24:05.353 INFO:teuthology.orchestra.run.trial038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-23T10:24:05.353 INFO:teuthology.orchestra.run.trial038.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-23T10:24:05.353 INFO:teuthology.orchestra.run.trial038.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-23T10:24:05.353 INFO:teuthology.orchestra.run.trial038.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-23T10:24:05.353 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-23T10:24:05.353 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-23T10:24:05.354 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-23T10:24:05.354 INFO:teuthology.orchestra.run.trial038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-23T10:24:05.354 INFO:teuthology.orchestra.run.trial038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-23T10:24:05.354 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:05.354 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:24:05.354 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:05.354 INFO:teuthology.orchestra.run.trial038.stdout:Install 10 Packages 2026-02-23T10:24:05.354 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:05.354 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 2.0 M 2026-02-23T10:24:05.354 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 25 M 2026-02-23T10:24:05.354 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:24:06.177 INFO:teuthology.orchestra.run.trial038.stdout:(1/10): ceph-mgr-rook-20.3.0-5439.ge371433f.el9 131 kB/s | 50 kB 00:00 2026-02-23T10:24:06.432 INFO:teuthology.orchestra.run.trial038.stdout:(2/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 249 kB/s | 159 kB 00:00 2026-02-23T10:24:06.521 INFO:teuthology.orchestra.run.trial038.stdout:(3/10): python3-pyasn1-modules-0.4.8-6.el9.noar 384 kB/s | 279 kB 00:00 2026-02-23T10:24:06.651 INFO:teuthology.orchestra.run.trial038.stdout:(4/10): python3-cachetools-4.2.4-1.el9.noarch.r 147 kB/s | 32 kB 00:00 2026-02-23T10:24:06.655 INFO:teuthology.orchestra.run.trial038.stdout:(5/10): python3-requests-oauthlib-1.3.0-12.el9. 112 kB/s | 54 kB 00:00 2026-02-23T10:24:06.662 INFO:teuthology.orchestra.run.trial038.stdout:(6/10): python3-certifi-2023.05.07-4.el9.noarch 100 kB/s | 14 kB 00:00 2026-02-23T10:24:06.943 INFO:teuthology.orchestra.run.trial038.stdout:(7/10): python3-rsa-4.9-2.el9.noarch.rpm 210 kB/s | 59 kB 00:00 2026-02-23T10:24:07.085 INFO:teuthology.orchestra.run.trial038.stdout:(8/10): python3-google-auth-2.45.0-1.el9.noarch 586 kB/s | 254 kB 00:00 2026-02-23T10:24:07.214 INFO:teuthology.orchestra.run.trial038.stdout:(9/10): python3-kubernetes-26.1.0-3.el9.noarch. 1.8 MB/s | 1.0 MB 00:00 2026-02-23T10:24:07.233 INFO:teuthology.orchestra.run.trial038.stdout:(10/10): python3-websocket-client-1.2.3-2.el9.n 309 kB/s | 90 kB 00:00 2026-02-23T10:24:07.235 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:07.235 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.1 MB/s | 2.0 MB 00:01 2026-02-23T10:24:07.261 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:24:07.270 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:24:07.270 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:24:07.311 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:24:07.311 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:24:07.387 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:24:07.438 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-23T10:24:07.450 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-23T10:24:07.465 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-23T10:24:07.469 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-23T10:24:07.477 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-23T10:24:07.516 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-23T10:24:07.528 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-23T10:24:07.845 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-23T10:24:07.860 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-23T10:24:07.861 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-rook-2:20.3.0-5439.ge371433f.el9.noarch 10/10 2026-02-23T10:24:07.968 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5439.ge371433f.el9.noarch 10/10 2026-02-23T10:24:07.968 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5439.ge371433f.el9.noarch 1/10 2026-02-23T10:24:07.968 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-23T10:24:07.968 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-23T10:24:07.968 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-23T10:24:07.968 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-23T10:24:07.968 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-23T10:24:07.968 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-23T10:24:07.969 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-23T10:24:07.969 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-23T10:24:08.025 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-23T10:24:08.025 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:08.025 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:24:08.025 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-rook-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:24:08.025 INFO:teuthology.orchestra.run.trial038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-23T10:24:08.025 INFO:teuthology.orchestra.run.trial038.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-23T10:24:08.026 INFO:teuthology.orchestra.run.trial038.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-23T10:24:08.026 INFO:teuthology.orchestra.run.trial038.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-23T10:24:08.026 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-23T10:24:08.026 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-23T10:24:08.026 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-23T10:24:08.026 INFO:teuthology.orchestra.run.trial038.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-23T10:24:08.026 INFO:teuthology.orchestra.run.trial038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-23T10:24:08.026 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:08.026 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:08.056 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-cephadm 2026-02-23T10:24:08.368 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:12 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:08.450 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:08.451 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:08.451 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-23T10:24:08.451 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:08.451 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:24:08.451 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 184 k 2026-02-23T10:24:08.451 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-23T10:24:08.452 INFO:teuthology.orchestra.run.trial038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-23T10:24:08.452 INFO:teuthology.orchestra.run.trial038.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-23T10:24:08.452 INFO:teuthology.orchestra.run.trial038.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-23T10:24:08.452 INFO:teuthology.orchestra.run.trial038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-23T10:24:08.452 INFO:teuthology.orchestra.run.trial038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-23T10:24:08.452 INFO:teuthology.orchestra.run.trial038.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-23T10:24:08.452 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:08.452 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:24:08.452 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:08.452 INFO:teuthology.orchestra.run.trial038.stdout:Install 7 Packages 2026-02-23T10:24:08.452 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:08.453 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 7.1 M 2026-02-23T10:24:08.453 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 34 M 2026-02-23T10:24:08.453 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:24:09.098 INFO:teuthology.orchestra.run.trial038.stdout:(1/7): ceph-mgr-cephadm-20.3.0-5439.ge371433f.e 389 kB/s | 184 kB 00:00 2026-02-23T10:24:09.136 INFO:teuthology.orchestra.run.trial038.stdout:(2/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 486 kB/s | 249 kB 00:00 2026-02-23T10:24:09.217 INFO:teuthology.orchestra.run.trial038.stdout:(3/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.6 MB/s | 548 kB 00:00 2026-02-23T10:24:09.236 INFO:teuthology.orchestra.run.trial038.stdout:(4/7): python3-natsort-7.1.1-5.el9.noarch.rpm 3.0 MB/s | 58 kB 00:00 2026-02-23T10:24:09.256 INFO:teuthology.orchestra.run.trial038.stdout:(5/7): python3-typing-extensions-4.15.0-1.el9.n 4.4 MB/s | 86 kB 00:00 2026-02-23T10:24:09.365 INFO:teuthology.orchestra.run.trial038.stdout:(6/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 130 kB/s | 35 kB 00:00 2026-02-23T10:24:09.503 INFO:teuthology.orchestra.run.trial038.stdout:(7/7): python3-babel-2.9.1-2.el9.noarch.rpm 6.8 MB/s | 6.0 MB 00:00 2026-02-23T10:24:09.503 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:09.503 INFO:teuthology.orchestra.run.trial038.stdout:Total 6.7 MB/s | 7.1 MB 00:01 2026-02-23T10:24:09.540 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:24:09.546 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:24:09.547 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:24:09.618 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:24:09.619 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:24:09.712 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:24:09.762 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-23T10:24:09.772 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-23T10:24:09.777 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-23T10:24:10.000 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-23T10:24:10.024 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-23T10:24:10.048 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-23T10:24:10.049 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5439.ge371433f.el9.noarch 7/7 2026-02-23T10:24:10.157 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5439.ge371433f.el9.noarch 7/7 2026-02-23T10:24:10.158 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5439.ge371433f.el9.noarch 1/7 2026-02-23T10:24:10.158 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-23T10:24:10.158 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-23T10:24:10.158 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-23T10:24:10.158 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-23T10:24:10.158 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-23T10:24:10.216 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-23T10:24:10.217 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:10.217 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:24:10.217 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-cephadm-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:24:10.217 INFO:teuthology.orchestra.run.trial038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-23T10:24:10.217 INFO:teuthology.orchestra.run.trial038.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-23T10:24:10.217 INFO:teuthology.orchestra.run.trial038.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-23T10:24:10.217 INFO:teuthology.orchestra.run.trial038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-23T10:24:10.217 INFO:teuthology.orchestra.run.trial038.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-23T10:24:10.217 INFO:teuthology.orchestra.run.trial038.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-23T10:24:10.217 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:10.218 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:10.248 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd 2026-02-23T10:24:10.556 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:14 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:10.620 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:10.638 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:10.639 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:24:10.639 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:10.659 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd-classic 2026-02-23T10:24:10.963 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:14 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:11.027 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:11.045 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:11.046 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:24:11.046 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:11.067 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-fuse 2026-02-23T10:24:11.377 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:15 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:11.459 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:11.460 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:11.460 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-23T10:24:11.460 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:11.460 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:24:11.460 INFO:teuthology.orchestra.run.trial038.stdout: ceph-fuse x86_64 2:20.3.0-5439.ge371433f.el9 ceph 914 k 2026-02-23T10:24:11.460 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:11.460 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:24:11.460 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:11.461 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-23T10:24:11.461 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:11.461 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 914 k 2026-02-23T10:24:11.461 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 2.9 M 2026-02-23T10:24:11.461 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:24:12.155 INFO:teuthology.orchestra.run.trial038.stdout:ceph-fuse-20.3.0-5439.ge371433f.el9.x86_64.rpm 1.3 MB/s | 914 kB 00:00 2026-02-23T10:24:12.155 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:12.155 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.3 MB/s | 914 kB 00:00 2026-02-23T10:24:12.156 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:24:12.160 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:24:12.160 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:24:12.193 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:24:12.193 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:24:12.284 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:24:12.301 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:12.520 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:12.578 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:12.578 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:12.578 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:24:12.579 INFO:teuthology.orchestra.run.trial038.stdout: ceph-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:12.579 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:12.579 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:12.606 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-volume 2026-02-23T10:24:12.920 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:16 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:13.002 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:13.003 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:13.003 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-23T10:24:13.004 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:13.004 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:24:13.004 INFO:teuthology.orchestra.run.trial038.stdout: ceph-volume noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 301 k 2026-02-23T10:24:13.004 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-23T10:24:13.004 INFO:teuthology.orchestra.run.trial038.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-23T10:24:13.004 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:13.004 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:24:13.004 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:13.004 INFO:teuthology.orchestra.run.trial038.stdout:Install 2 Packages 2026-02-23T10:24:13.004 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:13.004 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 652 k 2026-02-23T10:24:13.005 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 2.2 M 2026-02-23T10:24:13.005 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:24:13.322 INFO:teuthology.orchestra.run.trial038.stdout:(1/2): cryptsetup-2.8.1-3.el9.x86_64.rpm 6.0 MB/s | 351 kB 00:00 2026-02-23T10:24:13.781 INFO:teuthology.orchestra.run.trial038.stdout:(2/2): ceph-volume-20.3.0-5439.ge371433f.el9.no 585 kB/s | 301 kB 00:00 2026-02-23T10:24:13.781 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:13.781 INFO:teuthology.orchestra.run.trial038.stdout:Total 839 kB/s | 652 kB 00:00 2026-02-23T10:24:13.791 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:24:13.797 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:24:13.797 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:24:13.840 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:24:13.840 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:24:13.921 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:24:13.983 INFO:teuthology.orchestra.run.trial038.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 1/2 2026-02-23T10:24:13.985 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-volume-2:20.3.0-5439.ge371433f.el9.noarch 2/2 2026-02-23T10:24:14.000 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-volume-2:20.3.0-5439.ge371433f.el9.noarch 2/2 2026-02-23T10:24:14.000 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:24:14.001 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-23T10:24:14.001 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:14.238 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-volume-2:20.3.0-5439.ge371433f.el9.noarch 1/2 2026-02-23T10:24:14.297 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 2/2 2026-02-23T10:24:14.297 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:14.297 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:24:14.297 INFO:teuthology.orchestra.run.trial038.stdout: ceph-volume-2:20.3.0-5439.ge371433f.el9.noarch cryptsetup-2.8.1-3.el9.x86_64 2026-02-23T10:24:14.297 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:14.297 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:14.326 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados-devel 2026-02-23T10:24:14.635 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:18 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:14.718 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:14.719 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:14.719 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-23T10:24:14.719 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:14.719 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:24:14.719 INFO:teuthology.orchestra.run.trial038.stdout: librados-devel x86_64 2:20.3.0-5439.ge371433f.el9 ceph 124 k 2026-02-23T10:24:14.720 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:14.720 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:24:14.720 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:14.720 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-23T10:24:14.720 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:14.720 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 124 k 2026-02-23T10:24:14.720 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 434 k 2026-02-23T10:24:14.720 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:24:15.155 INFO:teuthology.orchestra.run.trial038.stdout:librados-devel-20.3.0-5439.ge371433f.el9.x86_64 284 kB/s | 124 kB 00:00 2026-02-23T10:24:15.156 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:15.156 INFO:teuthology.orchestra.run.trial038.stdout:Total 283 kB/s | 124 kB 00:00 2026-02-23T10:24:15.156 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:24:15.159 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:24:15.159 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:24:15.190 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:24:15.190 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:24:15.243 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:24:15.259 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librados-devel-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:15.347 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librados-devel-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:15.405 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librados-devel-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:15.405 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:15.406 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:24:15.406 INFO:teuthology.orchestra.run.trial038.stdout: librados-devel-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:15.406 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:15.406 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:15.434 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install libcephfs2 2026-02-23T10:24:15.742 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:19 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:15.806 INFO:teuthology.orchestra.run.trial038.stdout:Package libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:15.824 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:15.825 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:24:15.825 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:15.846 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install libcephfs-devel 2026-02-23T10:24:16.152 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:20 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:16.234 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:16.235 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:16.235 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-23T10:24:16.235 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:16.235 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:24:16.235 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-devel x86_64 2:20.3.0-5439.ge371433f.el9 ceph 37 k 2026-02-23T10:24:16.236 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-23T10:24:16.236 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 26 k 2026-02-23T10:24:16.236 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:16.236 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:24:16.236 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:16.236 INFO:teuthology.orchestra.run.trial038.stdout:Install 2 Packages 2026-02-23T10:24:16.236 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:16.236 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 64 k 2026-02-23T10:24:16.236 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 231 k 2026-02-23T10:24:16.236 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:24:16.589 INFO:teuthology.orchestra.run.trial038.stdout:(1/2): libcephfs-proxy2-20.3.0-5439.ge371433f.e 74 kB/s | 26 kB 00:00 2026-02-23T10:24:16.590 INFO:teuthology.orchestra.run.trial038.stdout:(2/2): libcephfs-devel-20.3.0-5439.ge371433f.el 105 kB/s | 37 kB 00:00 2026-02-23T10:24:16.591 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:16.591 INFO:teuthology.orchestra.run.trial038.stdout:Total 178 kB/s | 64 kB 00:00 2026-02-23T10:24:16.591 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:24:16.594 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:24:16.595 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:24:16.626 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:24:16.626 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:24:16.675 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:24:16.676 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephfs-proxy2-2:20.3.0-5439.ge371433f.el9.x86_64 1/2 2026-02-23T10:24:16.693 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5439.ge371433f.el9.x86_64 1/2 2026-02-23T10:24:16.710 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephfs-devel-2:20.3.0-5439.ge371433f.el9.x86_64 2/2 2026-02-23T10:24:16.765 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5439.ge371433f.el9.x86_64 2/2 2026-02-23T10:24:16.765 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephfs-devel-2:20.3.0-5439.ge371433f.el9.x86_64 1/2 2026-02-23T10:24:16.823 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5439.ge371433f.el9.x86_64 2/2 2026-02-23T10:24:16.823 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:16.823 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:24:16.823 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-devel-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:16.823 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-proxy2-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:16.823 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:16.823 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:16.850 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados2 2026-02-23T10:24:17.161 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:21 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:17.226 INFO:teuthology.orchestra.run.trial038.stdout:Package librados2-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:17.244 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:17.245 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:24:17.245 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:17.266 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librbd1 2026-02-23T10:24:17.577 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:21 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:17.642 INFO:teuthology.orchestra.run.trial038.stdout:Package librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:17.660 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:17.661 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:24:17.661 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:17.682 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rados 2026-02-23T10:24:17.994 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:21 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:18.058 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rados-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:18.076 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:18.076 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:24:18.077 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:18.097 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rgw 2026-02-23T10:24:18.182 INFO:teuthology.orchestra.run.trial153.stdout:(13/13): ceph-mgr-dashboard-20.3.0-5439.ge37143 2.3 MB/s | 51 MB 00:21 2026-02-23T10:24:18.184 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:18.184 INFO:teuthology.orchestra.run.trial153.stdout:Total 2.5 MB/s | 55 MB 00:22 2026-02-23T10:24:18.209 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:24:18.219 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:24:18.219 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:24:18.411 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:22 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:18.429 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:24:18.430 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:24:18.475 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rgw-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:18.493 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:18.494 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:24:18.494 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:18.515 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-cephfs 2026-02-23T10:24:18.656 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:24:18.662 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 1/13 2026-02-23T10:24:18.673 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 2/13 2026-02-23T10:24:18.675 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 3/13 2026-02-23T10:24:18.702 INFO:teuthology.orchestra.run.trial153.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 4/13 2026-02-23T10:24:18.745 INFO:teuthology.orchestra.run.trial153.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 5/13 2026-02-23T10:24:18.753 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 6/13 2026-02-23T10:24:18.781 INFO:teuthology.orchestra.run.trial153.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 7/13 2026-02-23T10:24:18.790 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 8/13 2026-02-23T10:24:18.797 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/13 2026-02-23T10:24:18.800 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 10/13 2026-02-23T10:24:18.809 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5439.ge371433f.el9 11/13 2026-02-23T10:24:18.824 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:22 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:18.889 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-cephfs-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:18.908 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:18.908 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:24:18.908 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:18.929 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rbd 2026-02-23T10:24:19.242 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:23 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:19.305 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rbd-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:19.324 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:19.324 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:24:19.324 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:19.345 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-fuse 2026-02-23T10:24:19.662 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:23 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:19.746 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:19.746 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:19.747 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-23T10:24:19.747 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:19.747 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:24:19.747 INFO:teuthology.orchestra.run.trial038.stdout: rbd-fuse x86_64 2:20.3.0-5439.ge371433f.el9 ceph 86 k 2026-02-23T10:24:19.747 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:19.747 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:24:19.747 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:19.747 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-23T10:24:19.747 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:19.747 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 86 k 2026-02-23T10:24:19.748 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 236 k 2026-02-23T10:24:19.748 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:24:20.094 INFO:teuthology.orchestra.run.trial038.stdout:rbd-fuse-20.3.0-5439.ge371433f.el9.x86_64.rpm 249 kB/s | 86 kB 00:00 2026-02-23T10:24:20.095 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:20.095 INFO:teuthology.orchestra.run.trial038.stdout:Total 248 kB/s | 86 kB 00:00 2026-02-23T10:24:20.095 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:24:20.099 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:24:20.099 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:24:20.133 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:24:20.133 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:24:20.184 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:24:20.200 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:20.285 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:20.344 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:20.344 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:20.344 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:24:20.344 INFO:teuthology.orchestra.run.trial038.stdout: rbd-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:20.344 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:20.344 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:20.372 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-mirror 2026-02-23T10:24:20.684 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:24 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:20.768 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:20.769 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:20.769 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-23T10:24:20.769 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:20.769 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:24:20.769 INFO:teuthology.orchestra.run.trial038.stdout: rbd-mirror x86_64 2:20.3.0-5439.ge371433f.el9 ceph 2.6 M 2026-02-23T10:24:20.769 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:20.769 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:24:20.769 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:20.769 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-23T10:24:20.769 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:20.770 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 2.6 M 2026-02-23T10:24:20.770 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 11 M 2026-02-23T10:24:20.770 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:24:21.669 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5439.ge371433f.el 12/13 2026-02-23T10:24:21.676 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5439.ge371433f.el9.noa 13/13 2026-02-23T10:24:21.935 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5439.ge371433f.el9.noa 13/13 2026-02-23T10:24:21.935 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5439.ge371433f.el 1/13 2026-02-23T10:24:21.935 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5439.ge371433f.el9.noa 2/13 2026-02-23T10:24:21.935 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5439.ge371433f.el9 3/13 2026-02-23T10:24:21.935 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 4/13 2026-02-23T10:24:21.935 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-23T10:24:21.935 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-23T10:24:21.936 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-23T10:24:21.936 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-23T10:24:21.936 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-23T10:24:21.936 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-23T10:24:21.936 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-23T10:24:21.936 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-23T10:24:21.990 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-23T10:24:21.991 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:21.991 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:24:21.991 INFO:teuthology.orchestra.run.trial153.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-23T10:24:21.991 INFO:teuthology.orchestra.run.trial153.stdout: ceph-grafana-dashboards-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:24:21.991 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr-dashboard-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:24:21.991 INFO:teuthology.orchestra.run.trial153.stdout: ceph-prometheus-alerts-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:24:21.991 INFO:teuthology.orchestra.run.trial153.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-23T10:24:21.991 INFO:teuthology.orchestra.run.trial153.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-23T10:24:21.991 INFO:teuthology.orchestra.run.trial153.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-23T10:24:21.991 INFO:teuthology.orchestra.run.trial153.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-23T10:24:21.991 INFO:teuthology.orchestra.run.trial153.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-23T10:24:21.992 INFO:teuthology.orchestra.run.trial153.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-23T10:24:21.992 INFO:teuthology.orchestra.run.trial153.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-23T10:24:21.992 INFO:teuthology.orchestra.run.trial153.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-23T10:24:21.992 INFO:teuthology.orchestra.run.trial153.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-23T10:24:21.992 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:21.992 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:22.028 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-23T10:24:22.341 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:28 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:22.381 INFO:teuthology.orchestra.run.trial038.stdout:rbd-mirror-20.3.0-5439.ge371433f.el9.x86_64.rpm 1.6 MB/s | 2.6 MB 00:01 2026-02-23T10:24:22.382 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:22.382 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.6 MB/s | 2.6 MB 00:01 2026-02-23T10:24:22.382 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:24:22.387 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:24:22.387 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:24:22.424 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:22.425 INFO:teuthology.orchestra.run.trial153.stdout:======================================================================================= 2026-02-23T10:24:22.425 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repository Size 2026-02-23T10:24:22.425 INFO:teuthology.orchestra.run.trial153.stdout:======================================================================================= 2026-02-23T10:24:22.425 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:24:22.425 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 7.4 M 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout:Installing dependencies: 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-23T10:24:22.426 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:22.427 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:24:22.427 INFO:teuthology.orchestra.run.trial153.stdout:======================================================================================= 2026-02-23T10:24:22.427 INFO:teuthology.orchestra.run.trial153.stdout:Install 12 Packages 2026-02-23T10:24:22.427 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:22.427 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 43 M 2026-02-23T10:24:22.427 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 234 M 2026-02-23T10:24:22.427 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:24:22.428 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:24:22.428 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:24:22.587 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:24:22.588 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-mirror-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:22.612 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:22.612 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:24:22.612 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-23T10:24:22.612 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-23T10:24:22.612 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-23T10:24:22.613 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:22.942 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-mirror-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:22.942 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:22.942 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:24:22.942 INFO:teuthology.orchestra.run.trial038.stdout: rbd-mirror-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:22.943 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:22.943 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:22.971 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-nbd 2026-02-23T10:24:23.283 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:27 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:23.340 INFO:teuthology.orchestra.run.trial153.stdout:(1/12): libquadmath-11.5.0-14.el9.x86_64.rpm 273 kB/s | 184 kB 00:00 2026-02-23T10:24:23.367 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:23.367 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:23.367 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-23T10:24:23.367 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:23.367 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:24:23.368 INFO:teuthology.orchestra.run.trial038.stdout: rbd-nbd x86_64 2:20.3.0-5439.ge371433f.el9 ceph 166 k 2026-02-23T10:24:23.368 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:23.368 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:24:23.368 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:23.368 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-23T10:24:23.368 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:23.368 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 166 k 2026-02-23T10:24:23.368 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 496 k 2026-02-23T10:24:23.368 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:24:23.369 INFO:teuthology.orchestra.run.trial153.stdout:(2/12): flexiblas-3.0.4-9.el9.x86_64.rpm 1.1 MB/s | 30 kB 00:00 2026-02-23T10:24:23.427 INFO:teuthology.orchestra.run.trial153.stdout:(3/12): flexiblas-netlib-3.0.4-9.el9.x86_64.rpm 50 MB/s | 3.0 MB 00:00 2026-02-23T10:24:23.436 INFO:teuthology.orchestra.run.trial153.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-9.el9.x 1.7 MB/s | 15 kB 00:00 2026-02-23T10:24:23.445 INFO:teuthology.orchestra.run.trial153.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 4.7 MB/s | 42 kB 00:00 2026-02-23T10:24:23.472 INFO:teuthology.orchestra.run.trial153.stdout:(6/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 192 MB/s | 5.3 MB 00:00 2026-02-23T10:24:23.484 INFO:teuthology.orchestra.run.trial153.stdout:(7/12): python3-devel-3.9.25-3.el9.x86_64.rpm 20 MB/s | 244 kB 00:00 2026-02-23T10:24:23.525 INFO:teuthology.orchestra.run.trial153.stdout:(8/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 149 MB/s | 6.1 MB 00:00 2026-02-23T10:24:23.538 INFO:teuthology.orchestra.run.trial153.stdout:(9/12): python3-numpy-f2py-1.23.5-2.el9.x86_64. 35 MB/s | 442 kB 00:00 2026-02-23T10:24:23.626 INFO:teuthology.orchestra.run.trial153.stdout:(10/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 219 MB/s | 19 MB 00:00 2026-02-23T10:24:23.791 INFO:teuthology.orchestra.run.trial038.stdout:rbd-nbd-20.3.0-5439.ge371433f.el9.x86_64.rpm 394 kB/s | 166 kB 00:00 2026-02-23T10:24:23.791 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:23.791 INFO:teuthology.orchestra.run.trial038.stdout:Total 393 kB/s | 166 kB 00:00 2026-02-23T10:24:23.791 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:24:23.796 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:24:23.796 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:24:23.827 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:24:23.827 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:24:23.878 INFO:teuthology.orchestra.run.trial153.stdout:(11/12): libgfortran-11.5.0-14.el9.x86_64.rpm 654 kB/s | 794 kB 00:01 2026-02-23T10:24:23.882 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:24:23.899 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-nbd-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:23.980 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:24.040 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-nbd-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:24.040 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:24.040 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:24:24.040 INFO:teuthology.orchestra.run.trial038.stdout: rbd-nbd-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:24.040 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:24.040 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:24.070 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd-crimson 2026-02-23T10:24:24.380 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:28 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:24.444 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-osd-crimson-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:24.462 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:24.463 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:24:24.463 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:24:24.485 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-23T10:24:24.796 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:28 ago on Mon 23 Feb 2026 10:20:56 AM UTC. 2026-02-23T10:24:24.880 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:24:24.880 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:24.880 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-23T10:24:24.880 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:24.880 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-23T10:24:24.880 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5439.ge371433f.el9 ceph 429 M 2026-02-23T10:24:24.880 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:24.881 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-23T10:24:24.881 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-23T10:24:24.881 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-23T10:24:24.881 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:24:24.881 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 429 M 2026-02-23T10:24:24.881 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 2.6 G 2026-02-23T10:24:24.881 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-23T10:24:27.000 INFO:teuthology.orchestra.run.trial153.stdout:(12/12): ceph-mgr-diskprediction-local-20.3.0-5 1.7 MB/s | 7.4 MB 00:04 2026-02-23T10:24:27.002 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:27.002 INFO:teuthology.orchestra.run.trial153.stdout:Total 9.3 MB/s | 43 MB 00:04 2026-02-23T10:24:27.163 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:24:27.176 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:24:27.177 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:24:27.376 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:24:27.376 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:24:27.590 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:24:27.593 INFO:teuthology.orchestra.run.trial153.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 1/12 2026-02-23T10:24:27.599 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-23T10:24:27.632 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-23T10:24:27.636 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 4/12 2026-02-23T10:24:27.694 INFO:teuthology.orchestra.run.trial153.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-23T10:24:27.696 INFO:teuthology.orchestra.run.trial153.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-23T10:24:27.717 INFO:teuthology.orchestra.run.trial153.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 7/12 2026-02-23T10:24:28.031 INFO:teuthology.orchestra.run.trial153.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 8/12 2026-02-23T10:24:28.103 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-23T10:24:28.754 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-23T10:24:29.207 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-23T10:24:29.207 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5439.ge3714 12/12 2026-02-23T10:24:29.479 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5439.ge3714 12/12 2026-02-23T10:24:29.479 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5439.ge3714 1/12 2026-02-23T10:24:29.479 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-23T10:24:29.479 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-23T10:24:29.480 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 4/12 2026-02-23T10:24:29.480 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 5/12 2026-02-23T10:24:29.480 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 6/12 2026-02-23T10:24:29.480 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-23T10:24:29.480 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-23T10:24:29.480 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 9/12 2026-02-23T10:24:29.480 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-23T10:24:29.480 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-23T10:24:29.536 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-23T10:24:29.536 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-23T10:24:29.537 INFO:teuthology.orchestra.run.trial153.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-23T10:24:29.538 INFO:teuthology.orchestra.run.trial153.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-23T10:24:29.538 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:29.538 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:29.575 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-mgr-rook 2026-02-23T10:24:29.886 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:35 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:29.971 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:29.972 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:29.972 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repository Size 2026-02-23T10:24:29.972 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:29.972 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:24:29.972 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr-rook noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 50 k 2026-02-23T10:24:29.972 INFO:teuthology.orchestra.run.trial153.stdout:Installing dependencies: 2026-02-23T10:24:29.972 INFO:teuthology.orchestra.run.trial153.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-23T10:24:29.972 INFO:teuthology.orchestra.run.trial153.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-23T10:24:29.972 INFO:teuthology.orchestra.run.trial153.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-23T10:24:29.972 INFO:teuthology.orchestra.run.trial153.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-23T10:24:29.973 INFO:teuthology.orchestra.run.trial153.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-23T10:24:29.973 INFO:teuthology.orchestra.run.trial153.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-23T10:24:29.973 INFO:teuthology.orchestra.run.trial153.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-23T10:24:29.973 INFO:teuthology.orchestra.run.trial153.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-23T10:24:29.973 INFO:teuthology.orchestra.run.trial153.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-23T10:24:29.973 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:29.973 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:24:29.973 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:29.973 INFO:teuthology.orchestra.run.trial153.stdout:Install 10 Packages 2026-02-23T10:24:29.973 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:29.973 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 2.0 M 2026-02-23T10:24:29.974 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 25 M 2026-02-23T10:24:29.974 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:24:30.558 INFO:teuthology.orchestra.run.trial153.stdout:(1/10): ceph-mgr-rook-20.3.0-5439.ge371433f.el9 125 kB/s | 50 kB 00:00 2026-02-23T10:24:30.876 INFO:teuthology.orchestra.run.trial153.stdout:(2/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 220 kB/s | 159 kB 00:00 2026-02-23T10:24:30.934 INFO:teuthology.orchestra.run.trial153.stdout:(3/10): python3-pyasn1-modules-0.4.8-6.el9.noar 358 kB/s | 279 kB 00:00 2026-02-23T10:24:31.080 INFO:teuthology.orchestra.run.trial153.stdout:(4/10): python3-certifi-2023.05.07-4.el9.noarch 97 kB/s | 14 kB 00:00 2026-02-23T10:24:31.085 INFO:teuthology.orchestra.run.trial153.stdout:(5/10): python3-cachetools-4.2.4-1.el9.noarch.r 154 kB/s | 32 kB 00:00 2026-02-23T10:24:31.155 INFO:teuthology.orchestra.run.trial153.stdout:(6/10): python3-requests-oauthlib-1.3.0-12.el9. 90 kB/s | 54 kB 00:00 2026-02-23T10:24:31.442 INFO:teuthology.orchestra.run.trial153.stdout:(7/10): python3-rsa-4.9-2.el9.noarch.rpm 206 kB/s | 59 kB 00:00 2026-02-23T10:24:31.496 INFO:teuthology.orchestra.run.trial153.stdout:(8/10): python3-google-auth-2.45.0-1.el9.noarch 612 kB/s | 254 kB 00:00 2026-02-23T10:24:31.662 INFO:teuthology.orchestra.run.trial153.stdout:(9/10): python3-kubernetes-26.1.0-3.el9.noarch. 1.8 MB/s | 1.0 MB 00:00 2026-02-23T10:24:31.724 INFO:teuthology.orchestra.run.trial153.stdout:(10/10): python3-websocket-client-1.2.3-2.el9.n 318 kB/s | 90 kB 00:00 2026-02-23T10:24:31.724 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:31.725 INFO:teuthology.orchestra.run.trial153.stdout:Total 1.1 MB/s | 2.0 MB 00:01 2026-02-23T10:24:31.751 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:24:31.759 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:24:31.759 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:24:31.798 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:24:31.799 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:24:31.873 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:24:31.924 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-23T10:24:31.937 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-23T10:24:31.951 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-23T10:24:31.956 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-23T10:24:31.964 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-23T10:24:32.003 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-23T10:24:32.015 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-23T10:24:32.338 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-23T10:24:32.353 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-23T10:24:32.353 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-mgr-rook-2:20.3.0-5439.ge371433f.el9.noarch 10/10 2026-02-23T10:24:32.459 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5439.ge371433f.el9.noarch 10/10 2026-02-23T10:24:32.460 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5439.ge371433f.el9.noarch 1/10 2026-02-23T10:24:32.460 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-23T10:24:32.460 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-23T10:24:32.460 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-23T10:24:32.460 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-23T10:24:32.460 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-23T10:24:32.460 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-23T10:24:32.460 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-23T10:24:32.461 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-23T10:24:32.517 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-23T10:24:32.517 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:32.517 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:24:32.517 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr-rook-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:24:32.517 INFO:teuthology.orchestra.run.trial153.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-23T10:24:32.518 INFO:teuthology.orchestra.run.trial153.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-23T10:24:32.518 INFO:teuthology.orchestra.run.trial153.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-23T10:24:32.518 INFO:teuthology.orchestra.run.trial153.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-23T10:24:32.518 INFO:teuthology.orchestra.run.trial153.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-23T10:24:32.518 INFO:teuthology.orchestra.run.trial153.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-23T10:24:32.518 INFO:teuthology.orchestra.run.trial153.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-23T10:24:32.518 INFO:teuthology.orchestra.run.trial153.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-23T10:24:32.518 INFO:teuthology.orchestra.run.trial153.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-23T10:24:32.518 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:32.518 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:32.547 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-mgr-cephadm 2026-02-23T10:24:32.853 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:38 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:32.934 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:32.935 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:32.936 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repository Size 2026-02-23T10:24:32.936 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:32.936 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:24:32.936 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 184 k 2026-02-23T10:24:32.936 INFO:teuthology.orchestra.run.trial153.stdout:Installing dependencies: 2026-02-23T10:24:32.936 INFO:teuthology.orchestra.run.trial153.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-23T10:24:32.936 INFO:teuthology.orchestra.run.trial153.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-23T10:24:32.936 INFO:teuthology.orchestra.run.trial153.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-23T10:24:32.936 INFO:teuthology.orchestra.run.trial153.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-23T10:24:32.936 INFO:teuthology.orchestra.run.trial153.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-23T10:24:32.937 INFO:teuthology.orchestra.run.trial153.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-23T10:24:32.937 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:32.937 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:24:32.937 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:32.937 INFO:teuthology.orchestra.run.trial153.stdout:Install 7 Packages 2026-02-23T10:24:32.937 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:32.937 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 7.1 M 2026-02-23T10:24:32.937 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 34 M 2026-02-23T10:24:32.937 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:24:33.774 INFO:teuthology.orchestra.run.trial153.stdout:(1/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 1.4 MB/s | 249 kB 00:00 2026-02-23T10:24:33.802 INFO:teuthology.orchestra.run.trial153.stdout:(2/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 1.2 MB/s | 35 kB 00:00 2026-02-23T10:24:33.884 INFO:teuthology.orchestra.run.trial153.stdout:(3/7): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2026-02-23T10:24:33.942 INFO:teuthology.orchestra.run.trial153.stdout:(4/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.8 MB/s | 548 kB 00:00 2026-02-23T10:24:33.953 INFO:teuthology.orchestra.run.trial153.stdout:(5/7): python3-natsort-7.1.1-5.el9.noarch.rpm 841 kB/s | 58 kB 00:00 2026-02-23T10:24:33.961 INFO:teuthology.orchestra.run.trial153.stdout:(6/7): python3-typing-extensions-4.15.0-1.el9.n 4.6 MB/s | 86 kB 00:00 2026-02-23T10:24:34.059 INFO:teuthology.orchestra.run.trial153.stdout:(7/7): ceph-mgr-cephadm-20.3.0-5439.ge371433f.e 397 kB/s | 184 kB 00:00 2026-02-23T10:24:34.059 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:34.060 INFO:teuthology.orchestra.run.trial153.stdout:Total 6.3 MB/s | 7.1 MB 00:01 2026-02-23T10:24:34.095 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:24:34.101 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:24:34.101 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:24:34.171 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:24:34.172 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:24:34.260 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:24:34.310 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-23T10:24:34.320 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-23T10:24:34.325 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-23T10:24:34.547 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-23T10:24:34.570 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-23T10:24:34.595 INFO:teuthology.orchestra.run.trial153.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-23T10:24:34.596 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5439.ge371433f.el9.noarch 7/7 2026-02-23T10:24:34.705 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5439.ge371433f.el9.noarch 7/7 2026-02-23T10:24:34.705 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5439.ge371433f.el9.noarch 1/7 2026-02-23T10:24:34.705 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-23T10:24:34.705 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-23T10:24:34.706 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-23T10:24:34.706 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-23T10:24:34.706 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-23T10:24:34.763 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-23T10:24:34.763 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:34.763 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:24:34.763 INFO:teuthology.orchestra.run.trial153.stdout: ceph-mgr-cephadm-2:20.3.0-5439.ge371433f.el9.noarch 2026-02-23T10:24:34.764 INFO:teuthology.orchestra.run.trial153.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-23T10:24:34.764 INFO:teuthology.orchestra.run.trial153.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-23T10:24:34.764 INFO:teuthology.orchestra.run.trial153.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-23T10:24:34.764 INFO:teuthology.orchestra.run.trial153.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-23T10:24:34.764 INFO:teuthology.orchestra.run.trial153.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-23T10:24:34.764 INFO:teuthology.orchestra.run.trial153.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-23T10:24:34.764 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:34.764 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:34.794 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-osd 2026-02-23T10:24:35.109 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:41 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:35.174 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:35.192 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:35.192 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:24:35.193 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:35.214 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-osd-classic 2026-02-23T10:24:35.518 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:41 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:35.583 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:35.601 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:35.602 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:24:35.602 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:35.623 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-fuse 2026-02-23T10:24:35.932 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:41 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:36.016 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:36.017 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:36.017 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repo Size 2026-02-23T10:24:36.017 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:36.017 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:24:36.017 INFO:teuthology.orchestra.run.trial153.stdout: ceph-fuse x86_64 2:20.3.0-5439.ge371433f.el9 ceph 914 k 2026-02-23T10:24:36.017 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:36.017 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:24:36.017 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:36.017 INFO:teuthology.orchestra.run.trial153.stdout:Install 1 Package 2026-02-23T10:24:36.017 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:36.018 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 914 k 2026-02-23T10:24:36.018 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 2.9 M 2026-02-23T10:24:36.018 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:24:36.948 INFO:teuthology.orchestra.run.trial153.stdout:ceph-fuse-20.3.0-5439.ge371433f.el9.x86_64.rpm 982 kB/s | 914 kB 00:00 2026-02-23T10:24:36.949 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:36.949 INFO:teuthology.orchestra.run.trial153.stdout:Total 980 kB/s | 914 kB 00:00 2026-02-23T10:24:36.949 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:24:36.954 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:24:36.954 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:24:36.986 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:24:36.987 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:24:37.076 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:24:37.093 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:37.327 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:37.385 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:37.385 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:37.385 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:24:37.385 INFO:teuthology.orchestra.run.trial153.stdout: ceph-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:37.385 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:37.385 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:37.413 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-volume 2026-02-23T10:24:37.728 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:43 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:37.810 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:37.811 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:37.811 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repository Size 2026-02-23T10:24:37.811 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:37.811 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:24:37.811 INFO:teuthology.orchestra.run.trial153.stdout: ceph-volume noarch 2:20.3.0-5439.ge371433f.el9 ceph-noarch 301 k 2026-02-23T10:24:37.811 INFO:teuthology.orchestra.run.trial153.stdout:Installing dependencies: 2026-02-23T10:24:37.812 INFO:teuthology.orchestra.run.trial153.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-23T10:24:37.812 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:37.812 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:24:37.812 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:37.812 INFO:teuthology.orchestra.run.trial153.stdout:Install 2 Packages 2026-02-23T10:24:37.812 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:37.812 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 652 k 2026-02-23T10:24:37.812 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 2.2 M 2026-02-23T10:24:37.812 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:24:37.902 INFO:teuthology.orchestra.run.trial153.stdout:(1/2): cryptsetup-2.8.1-3.el9.x86_64.rpm 11 MB/s | 351 kB 00:00 2026-02-23T10:24:38.493 INFO:teuthology.orchestra.run.trial153.stdout:(2/2): ceph-volume-20.3.0-5439.ge371433f.el9.no 484 kB/s | 301 kB 00:00 2026-02-23T10:24:38.494 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:38.494 INFO:teuthology.orchestra.run.trial153.stdout:Total 955 kB/s | 652 kB 00:00 2026-02-23T10:24:38.505 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:24:38.511 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:24:38.511 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:24:38.555 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:24:38.555 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:24:38.635 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:24:38.698 INFO:teuthology.orchestra.run.trial153.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 1/2 2026-02-23T10:24:38.700 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-volume-2:20.3.0-5439.ge371433f.el9.noarch 2/2 2026-02-23T10:24:38.715 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-volume-2:20.3.0-5439.ge371433f.el9.noarch 2/2 2026-02-23T10:24:38.715 INFO:teuthology.orchestra.run.trial153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:24:38.715 INFO:teuthology.orchestra.run.trial153.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-23T10:24:38.715 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:38.937 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-volume-2:20.3.0-5439.ge371433f.el9.noarch 1/2 2026-02-23T10:24:38.994 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 2/2 2026-02-23T10:24:38.994 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:38.994 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:24:38.994 INFO:teuthology.orchestra.run.trial153.stdout: ceph-volume-2:20.3.0-5439.ge371433f.el9.noarch cryptsetup-2.8.1-3.el9.x86_64 2026-02-23T10:24:38.994 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:38.995 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:39.022 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install librados-devel 2026-02-23T10:24:39.333 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:45 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:39.416 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repo Size 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout: librados-devel x86_64 2:20.3.0-5439.ge371433f.el9 ceph 124 k 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout:Install 1 Package 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 124 k 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 434 k 2026-02-23T10:24:39.417 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:24:39.884 INFO:teuthology.orchestra.run.trial153.stdout:librados-devel-20.3.0-5439.ge371433f.el9.x86_64 265 kB/s | 124 kB 00:00 2026-02-23T10:24:39.885 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:39.885 INFO:teuthology.orchestra.run.trial153.stdout:Total 264 kB/s | 124 kB 00:00 2026-02-23T10:24:39.885 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:24:39.888 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:24:39.888 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:24:39.918 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:24:39.919 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:24:39.972 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:24:39.988 INFO:teuthology.orchestra.run.trial153.stdout: Installing : librados-devel-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:40.072 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: librados-devel-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:40.130 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : librados-devel-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:40.130 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:40.131 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:24:40.131 INFO:teuthology.orchestra.run.trial153.stdout: librados-devel-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:40.131 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:40.131 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:40.159 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install libcephfs2 2026-02-23T10:24:40.470 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:46 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:40.534 INFO:teuthology.orchestra.run.trial153.stdout:Package libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:40.552 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:40.553 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:24:40.553 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:40.575 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install libcephfs-devel 2026-02-23T10:24:40.880 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:46 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:40.965 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:40.966 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:40.967 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repo Size 2026-02-23T10:24:40.967 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:40.967 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:24:40.967 INFO:teuthology.orchestra.run.trial153.stdout: libcephfs-devel x86_64 2:20.3.0-5439.ge371433f.el9 ceph 37 k 2026-02-23T10:24:40.967 INFO:teuthology.orchestra.run.trial153.stdout:Installing dependencies: 2026-02-23T10:24:40.967 INFO:teuthology.orchestra.run.trial153.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5439.ge371433f.el9 ceph 26 k 2026-02-23T10:24:40.967 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:40.967 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:24:40.967 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:40.967 INFO:teuthology.orchestra.run.trial153.stdout:Install 2 Packages 2026-02-23T10:24:40.967 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:40.968 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 64 k 2026-02-23T10:24:40.968 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 231 k 2026-02-23T10:24:40.968 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:24:41.311 INFO:teuthology.orchestra.run.trial153.stdout:(1/2): libcephfs-devel-20.3.0-5439.ge371433f.el 108 kB/s | 37 kB 00:00 2026-02-23T10:24:41.313 INFO:teuthology.orchestra.run.trial153.stdout:(2/2): libcephfs-proxy2-20.3.0-5439.ge371433f.e 76 kB/s | 26 kB 00:00 2026-02-23T10:24:41.314 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:41.314 INFO:teuthology.orchestra.run.trial153.stdout:Total 183 kB/s | 64 kB 00:00 2026-02-23T10:24:41.314 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:24:41.317 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:24:41.318 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:24:41.349 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:24:41.349 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:24:41.397 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:24:41.398 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libcephfs-proxy2-2:20.3.0-5439.ge371433f.el9.x86_64 1/2 2026-02-23T10:24:41.413 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5439.ge371433f.el9.x86_64 1/2 2026-02-23T10:24:41.429 INFO:teuthology.orchestra.run.trial153.stdout: Installing : libcephfs-devel-2:20.3.0-5439.ge371433f.el9.x86_64 2/2 2026-02-23T10:24:41.485 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5439.ge371433f.el9.x86_64 2/2 2026-02-23T10:24:41.485 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libcephfs-devel-2:20.3.0-5439.ge371433f.el9.x86_64 1/2 2026-02-23T10:24:41.543 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5439.ge371433f.el9.x86_64 2/2 2026-02-23T10:24:41.543 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:41.544 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:24:41.544 INFO:teuthology.orchestra.run.trial153.stdout: libcephfs-devel-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:41.544 INFO:teuthology.orchestra.run.trial153.stdout: libcephfs-proxy2-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:41.544 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:41.544 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:41.570 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install librados2 2026-02-23T10:24:41.878 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:47 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:41.943 INFO:teuthology.orchestra.run.trial153.stdout:Package librados2-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:41.961 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:41.962 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:24:41.962 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:41.983 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install librbd1 2026-02-23T10:24:42.291 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:48 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:42.356 INFO:teuthology.orchestra.run.trial153.stdout:Package librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:42.374 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:42.374 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:24:42.375 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:42.396 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install python3-rados 2026-02-23T10:24:42.706 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:48 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:42.771 INFO:teuthology.orchestra.run.trial153.stdout:Package python3-rados-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:42.789 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:42.790 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:24:42.790 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:42.811 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install python3-rgw 2026-02-23T10:24:43.124 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:49 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:43.189 INFO:teuthology.orchestra.run.trial153.stdout:Package python3-rgw-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:43.207 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:43.208 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:24:43.208 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:43.229 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install python3-cephfs 2026-02-23T10:24:43.535 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:49 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:43.600 INFO:teuthology.orchestra.run.trial153.stdout:Package python3-cephfs-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:43.618 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:43.618 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:24:43.619 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:43.640 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install python3-rbd 2026-02-23T10:24:43.948 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:49 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:44.012 INFO:teuthology.orchestra.run.trial153.stdout:Package python3-rbd-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:44.030 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:44.030 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:24:44.031 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:44.052 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install rbd-fuse 2026-02-23T10:24:44.370 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:50 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:44.454 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:44.455 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:44.455 INFO:teuthology.orchestra.run.trial153.stdout: Package Architecture Version Repository Size 2026-02-23T10:24:44.455 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:44.455 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:24:44.455 INFO:teuthology.orchestra.run.trial153.stdout: rbd-fuse x86_64 2:20.3.0-5439.ge371433f.el9 ceph 86 k 2026-02-23T10:24:44.455 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:44.456 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:24:44.456 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:44.456 INFO:teuthology.orchestra.run.trial153.stdout:Install 1 Package 2026-02-23T10:24:44.456 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:44.456 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 86 k 2026-02-23T10:24:44.456 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 236 k 2026-02-23T10:24:44.456 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:24:44.880 INFO:teuthology.orchestra.run.trial153.stdout:rbd-fuse-20.3.0-5439.ge371433f.el9.x86_64.rpm 204 kB/s | 86 kB 00:00 2026-02-23T10:24:44.880 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:44.880 INFO:teuthology.orchestra.run.trial153.stdout:Total 203 kB/s | 86 kB 00:00 2026-02-23T10:24:44.880 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:24:44.884 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:24:44.884 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:24:44.915 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:24:44.915 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:24:44.966 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:24:44.983 INFO:teuthology.orchestra.run.trial153.stdout: Installing : rbd-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:45.065 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:45.122 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : rbd-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:45.123 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:45.123 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:24:45.123 INFO:teuthology.orchestra.run.trial153.stdout: rbd-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:45.123 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:45.123 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:45.152 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install rbd-mirror 2026-02-23T10:24:45.459 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:51 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:45.542 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:45.542 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:45.542 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repo Size 2026-02-23T10:24:45.542 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:45.543 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:24:45.543 INFO:teuthology.orchestra.run.trial153.stdout: rbd-mirror x86_64 2:20.3.0-5439.ge371433f.el9 ceph 2.6 M 2026-02-23T10:24:45.543 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:45.543 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:24:45.543 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:45.543 INFO:teuthology.orchestra.run.trial153.stdout:Install 1 Package 2026-02-23T10:24:45.543 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:45.543 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 2.6 M 2026-02-23T10:24:45.543 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 11 M 2026-02-23T10:24:45.543 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:24:47.036 INFO:teuthology.orchestra.run.trial153.stdout:rbd-mirror-20.3.0-5439.ge371433f.el9.x86_64.rpm 1.7 MB/s | 2.6 MB 00:01 2026-02-23T10:24:47.036 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:47.036 INFO:teuthology.orchestra.run.trial153.stdout:Total 1.7 MB/s | 2.6 MB 00:01 2026-02-23T10:24:47.036 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:24:47.041 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:24:47.041 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:24:47.078 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:24:47.079 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:24:47.240 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:24:47.241 INFO:teuthology.orchestra.run.trial153.stdout: Installing : rbd-mirror-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:47.264 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:47.264 INFO:teuthology.orchestra.run.trial153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-23T10:24:47.264 INFO:teuthology.orchestra.run.trial153.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-23T10:24:47.264 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-23T10:24:47.264 INFO:teuthology.orchestra.run.trial153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-23T10:24:47.265 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:47.551 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : rbd-mirror-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:47.551 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:47.551 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:24:47.551 INFO:teuthology.orchestra.run.trial153.stdout: rbd-mirror-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:47.551 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:47.551 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:47.580 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install rbd-nbd 2026-02-23T10:24:47.893 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:53 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:47.977 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:47.978 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:47.978 INFO:teuthology.orchestra.run.trial153.stdout: Package Architecture Version Repository Size 2026-02-23T10:24:47.978 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:47.978 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:24:47.978 INFO:teuthology.orchestra.run.trial153.stdout: rbd-nbd x86_64 2:20.3.0-5439.ge371433f.el9 ceph 166 k 2026-02-23T10:24:47.979 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:47.979 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:24:47.979 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:47.979 INFO:teuthology.orchestra.run.trial153.stdout:Install 1 Package 2026-02-23T10:24:47.979 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:47.979 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 166 k 2026-02-23T10:24:47.979 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 496 k 2026-02-23T10:24:47.979 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:24:48.420 INFO:teuthology.orchestra.run.trial153.stdout:rbd-nbd-20.3.0-5439.ge371433f.el9.x86_64.rpm 378 kB/s | 166 kB 00:00 2026-02-23T10:24:48.420 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:24:48.421 INFO:teuthology.orchestra.run.trial153.stdout:Total 377 kB/s | 166 kB 00:00 2026-02-23T10:24:48.421 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:24:48.425 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:24:48.425 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:24:48.457 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:24:48.457 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:24:48.512 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:24:48.528 INFO:teuthology.orchestra.run.trial153.stdout: Installing : rbd-nbd-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:48.618 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:48.676 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : rbd-nbd-2:20.3.0-5439.ge371433f.el9.x86_64 1/1 2026-02-23T10:24:48.676 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:48.676 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:24:48.676 INFO:teuthology.orchestra.run.trial153.stdout: rbd-nbd-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:24:48.676 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:48.676 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:48.704 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-osd-crimson 2026-02-23T10:24:49.012 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:55 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:49.076 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-osd-crimson-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:24:49.095 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:49.096 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:24:49.096 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:24:49.117 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-23T10:24:49.430 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:03:55 ago on Mon 23 Feb 2026 10:20:54 AM UTC. 2026-02-23T10:24:49.513 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:24:49.513 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout: Package Arch Version Repo Size 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout:Installing: 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5439.ge371433f.el9 ceph 429 M 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout:Transaction Summary 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout:================================================================================ 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout:Install 1 Package 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout:Total download size: 429 M 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout:Installed size: 2.6 G 2026-02-23T10:24:49.514 INFO:teuthology.orchestra.run.trial153.stdout:Downloading Packages: 2026-02-23T10:25:49.687 INFO:teuthology.orchestra.run.trial038.stdout:ceph-osd-crimson-debuginfo-20.3.0-5439.ge371433 5.1 MB/s | 429 MB 01:24 2026-02-23T10:25:49.688 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:25:49.688 INFO:teuthology.orchestra.run.trial038.stdout:Total 5.1 MB/s | 429 MB 01:24 2026-02-23T10:25:49.688 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-23T10:25:49.690 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-23T10:25:49.690 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-23T10:25:50.975 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-23T10:25:50.975 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-23T10:26:14.398 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-23T10:26:14.414 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5439.ge371433f.e 1/1 2026-02-23T10:26:16.332 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5439.ge371433f.e 1/1 2026-02-23T10:26:16.390 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5439.ge371433f.e 1/1 2026-02-23T10:26:16.390 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:26:16.390 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-23T10:26:16.390 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:26:16.391 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:26:16.391 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:26:16.464 DEBUG:teuthology.parallel:result is None 2026-02-23T10:26:20.652 INFO:teuthology.orchestra.run.trial153.stdout:ceph-osd-crimson-debuginfo-20.3.0-5439.ge371433 4.7 MB/s | 429 MB 01:31 2026-02-23T10:26:20.653 INFO:teuthology.orchestra.run.trial153.stdout:-------------------------------------------------------------------------------- 2026-02-23T10:26:20.653 INFO:teuthology.orchestra.run.trial153.stdout:Total 4.7 MB/s | 429 MB 01:31 2026-02-23T10:26:20.653 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction check 2026-02-23T10:26:20.655 INFO:teuthology.orchestra.run.trial153.stdout:Transaction check succeeded. 2026-02-23T10:26:20.655 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction test 2026-02-23T10:26:21.938 INFO:teuthology.orchestra.run.trial153.stdout:Transaction test succeeded. 2026-02-23T10:26:21.939 INFO:teuthology.orchestra.run.trial153.stdout:Running transaction 2026-02-23T10:26:45.731 INFO:teuthology.orchestra.run.trial153.stdout: Preparing : 1/1 2026-02-23T10:26:45.747 INFO:teuthology.orchestra.run.trial153.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5439.ge371433f.e 1/1 2026-02-23T10:26:47.663 INFO:teuthology.orchestra.run.trial153.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5439.ge371433f.e 1/1 2026-02-23T10:26:47.721 INFO:teuthology.orchestra.run.trial153.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5439.ge371433f.e 1/1 2026-02-23T10:26:47.721 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:26:47.721 INFO:teuthology.orchestra.run.trial153.stdout:Installed: 2026-02-23T10:26:47.721 INFO:teuthology.orchestra.run.trial153.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5439.ge371433f.el9.x86_64 2026-02-23T10:26:47.721 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:26:47.721 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:26:47.792 DEBUG:teuthology.parallel:result is None 2026-02-23T10:26:47.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e371433fcd1390a0b82cfb18effc422c1f5d8c1d 2026-02-23T10:26:47.958 DEBUG:teuthology.orchestra.run.trial038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-23T10:26:47.981 INFO:teuthology.orchestra.run.trial038.stdout:20.3.0-5439.ge371433f.el9 2026-02-23T10:26:47.981 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5439.ge371433f.el9 2026-02-23T10:26:47.981 INFO:teuthology.task.install:The correct ceph version 20.3.0-5439.ge371433f is installed. 2026-02-23T10:26:47.984 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e371433fcd1390a0b82cfb18effc422c1f5d8c1d 2026-02-23T10:26:48.141 DEBUG:teuthology.orchestra.run.trial153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-23T10:26:48.162 INFO:teuthology.orchestra.run.trial153.stdout:20.3.0-5439.ge371433f.el9 2026-02-23T10:26:48.163 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5439.ge371433f.el9 2026-02-23T10:26:48.163 INFO:teuthology.task.install:The correct ceph version 20.3.0-5439.ge371433f is installed. 2026-02-23T10:26:48.166 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-23T10:26:48.166 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:26:48.166 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-23T10:26:48.193 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:26:48.194 DEBUG:teuthology.orchestra.run.trial153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-23T10:26:48.227 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-23T10:26:48.227 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:26:48.227 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/usr/bin/daemon-helper 2026-02-23T10:26:48.259 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-23T10:26:48.324 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:26:48.324 DEBUG:teuthology.orchestra.run.trial153:> sudo dd of=/usr/bin/daemon-helper 2026-02-23T10:26:48.350 DEBUG:teuthology.orchestra.run.trial153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-23T10:26:48.415 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-23T10:26:48.415 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:26:48.416 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-23T10:26:48.442 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-23T10:26:48.506 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:26:48.506 DEBUG:teuthology.orchestra.run.trial153:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-23T10:26:48.532 DEBUG:teuthology.orchestra.run.trial153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-23T10:26:48.594 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-23T10:26:48.595 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:26:48.595 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/usr/bin/stdin-killer 2026-02-23T10:26:48.621 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-23T10:26:48.685 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:26:48.686 DEBUG:teuthology.orchestra.run.trial153:> sudo dd of=/usr/bin/stdin-killer 2026-02-23T10:26:48.710 DEBUG:teuthology.orchestra.run.trial153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-23T10:26:48.774 INFO:teuthology.run_tasks:Running task pexec... 2026-02-23T10:26:48.789 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-23T10:26:48.790 DEBUG:teuthology.orchestra.run.trial038:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-23T10:26:48.791 DEBUG:teuthology.orchestra.run.trial153:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-23T10:26:48.794 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial038.front.sepia.ceph.com 2026-02-23T10:26:48.794 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-23T10:26:48.816 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial153.front.sepia.ceph.com 2026-02-23T10:26:48.816 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-23T10:26:48.844 DEBUG:teuthology.parallel:result is None 2026-02-23T10:26:48.866 DEBUG:teuthology.parallel:result is None 2026-02-23T10:26:48.867 INFO:teuthology.run_tasks:Running task install... 2026-02-23T10:26:48.882 DEBUG:teuthology.task.install:project ceph 2026-02-23T10:26:48.882 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d'}} 2026-02-23T10:26:48.882 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d'} 2026-02-23T10:26:48.882 INFO:teuthology.task.install:Using flavor: debug 2026-02-23T10:26:48.891 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-23T10:26:48.891 INFO:teuthology.task.install:extra packages: [] 2026-02-23T10:26:48.891 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d', 'tag': None, 'wait_for_package': False} 2026-02-23T10:26:48.892 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e371433fcd1390a0b82cfb18effc422c1f5d8c1d 2026-02-23T10:26:48.893 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d', 'tag': None, 'wait_for_package': False} 2026-02-23T10:26:48.894 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e371433fcd1390a0b82cfb18effc422c1f5d8c1d 2026-02-23T10:26:49.081 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/ 2026-02-23T10:26:49.081 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5439.ge371433f 2026-02-23T10:26:49.092 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/ 2026-02-23T10:26:49.092 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5439.ge371433f 2026-02-23T10:26:49.413 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-23T10:26:49.414 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:26:49.414 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-23T10:26:49.440 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-23T10:26:49.440 DEBUG:teuthology.orchestra.run.trial038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-23T10:26:49.463 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-23T10:26:49.463 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:26:49.463 DEBUG:teuthology.orchestra.run.trial153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-23T10:26:49.490 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-23T10:26:49.490 DEBUG:teuthology.orchestra.run.trial153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e371433fcd1390a0b82cfb18effc422c1f5d8c1d/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-23T10:26:49.505 DEBUG:teuthology.orchestra.run.trial038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-23T10:26:49.554 DEBUG:teuthology.orchestra.run.trial153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-23T10:26:49.569 DEBUG:teuthology.orchestra.run.trial038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-23T10:26:49.618 DEBUG:teuthology.orchestra.run.trial153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-23T10:26:49.625 INFO:teuthology.orchestra.run.trial038.stdout:check_obsoletes = 1 2026-02-23T10:26:49.635 DEBUG:teuthology.orchestra.run.trial038:> sudo yum clean all 2026-02-23T10:26:49.673 INFO:teuthology.orchestra.run.trial153.stdout:check_obsoletes = 1 2026-02-23T10:26:49.683 DEBUG:teuthology.orchestra.run.trial153:> sudo yum clean all 2026-02-23T10:26:49.796 INFO:teuthology.orchestra.run.trial038.stdout:56 files removed 2026-02-23T10:26:49.811 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-radosgw 2026-02-23T10:26:49.841 INFO:teuthology.orchestra.run.trial153.stdout:56 files removed 2026-02-23T10:26:49.856 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-radosgw 2026-02-23T10:26:50.669 INFO:teuthology.orchestra.run.trial038.stdout:ceph packages for x86_64 130 kB/s | 92 kB 00:00 2026-02-23T10:26:50.702 INFO:teuthology.orchestra.run.trial153.stdout:ceph packages for x86_64 131 kB/s | 92 kB 00:00 2026-02-23T10:26:51.303 INFO:teuthology.orchestra.run.trial038.stdout:ceph noarch packages 39 kB/s | 24 kB 00:00 2026-02-23T10:26:51.328 INFO:teuthology.orchestra.run.trial153.stdout:ceph noarch packages 39 kB/s | 24 kB 00:00 2026-02-23T10:26:51.850 INFO:teuthology.orchestra.run.trial038.stdout:ceph source packages 3.9 kB/s | 2.1 kB 00:00 2026-02-23T10:26:51.881 INFO:teuthology.orchestra.run.trial153.stdout:ceph source packages 3.9 kB/s | 2.1 kB 00:00 2026-02-23T10:26:52.546 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-23T10:26:52.615 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.9 MB 00:00 2026-02-23T10:26:55.593 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-23T10:26:55.838 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-23T10:27:00.270 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 7.8 MB 00:02 2026-02-23T10:27:02.138 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 20 kB 00:01 2026-02-23T10:27:02.842 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 7.8 MB 00:04 2026-02-23T10:27:03.410 INFO:teuthology.orchestra.run.trial038.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2026-02-23T10:27:04.581 INFO:teuthology.orchestra.run.trial153.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 20 kB 00:01 2026-02-23T10:27:04.922 INFO:teuthology.orchestra.run.trial153.stdout:Extra Packages for Enterprise Linux 73 MB/s | 20 MB 00:00 2026-02-23T10:27:06.673 INFO:teuthology.orchestra.run.trial038.stdout:lab-extras 1.1 MB/s | 50 kB 00:00 2026-02-23T10:27:07.863 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-radosgw-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:07.882 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:07.883 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:07.883 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:07.906 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-test 2026-02-23T10:27:08.158 INFO:teuthology.orchestra.run.trial153.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-23T10:27:08.218 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:02 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:08.282 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-test-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:08.301 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:08.302 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:08.302 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:08.324 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph 2026-02-23T10:27:08.637 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:02 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:08.701 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:08.719 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:08.720 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:08.720 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:08.741 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-base 2026-02-23T10:27:09.048 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:03 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:09.113 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:09.131 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:09.132 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:09.132 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:09.153 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install cephadm 2026-02-23T10:27:09.337 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-radosgw-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:09.356 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:09.356 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:09.356 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:09.381 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-test 2026-02-23T10:27:09.466 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:03 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:09.531 INFO:teuthology.orchestra.run.trial038.stdout:Package cephadm-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:09.549 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:09.550 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:09.550 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:09.571 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-immutable-object-cache 2026-02-23T10:27:09.695 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:01 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:09.761 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-test-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:09.779 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:09.779 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:09.780 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:09.801 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph 2026-02-23T10:27:09.884 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:03 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:09.948 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-immutable-object-cache-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:09.967 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:09.967 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:09.968 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:09.989 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr 2026-02-23T10:27:10.113 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:02 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:10.178 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:10.196 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:10.197 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:10.197 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:10.219 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-base 2026-02-23T10:27:10.304 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:04 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:10.369 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:10.388 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:10.388 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:10.388 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:10.410 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-dashboard 2026-02-23T10:27:10.530 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:02 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:10.596 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-base-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:10.614 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:10.615 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:10.615 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:10.636 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install cephadm 2026-02-23T10:27:10.725 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:04 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:10.789 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-dashboard-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:10.807 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:10.808 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:10.808 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:10.829 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-23T10:27:10.948 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:02 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:11.014 INFO:teuthology.orchestra.run.trial153.stdout:Package cephadm-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:11.032 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:11.033 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:11.033 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:11.055 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-immutable-object-cache 2026-02-23T10:27:11.143 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:05 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:11.208 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:11.226 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:11.227 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:11.227 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:11.249 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-rook 2026-02-23T10:27:11.370 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:03 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:11.434 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-immutable-object-cache-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:11.453 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:11.454 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:11.454 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:11.476 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-mgr 2026-02-23T10:27:11.559 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:05 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:11.624 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-rook-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:11.643 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:11.644 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:11.644 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:11.667 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-cephadm 2026-02-23T10:27:11.786 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:03 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:11.852 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-mgr-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:11.870 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:11.871 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:11.871 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:11.894 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-mgr-dashboard 2026-02-23T10:27:11.974 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:05 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:12.038 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-cephadm-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:12.057 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:12.057 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:12.058 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:12.079 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd 2026-02-23T10:27:12.208 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:04 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:12.274 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-mgr-dashboard-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:12.292 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:12.293 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:12.293 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:12.315 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-23T10:27:12.396 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:06 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:12.461 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:12.479 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:12.480 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:12.480 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:12.502 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd-classic 2026-02-23T10:27:12.629 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:04 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:12.693 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:12.712 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:12.712 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:12.713 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:12.734 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-mgr-rook 2026-02-23T10:27:12.812 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:06 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:12.878 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:12.897 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:12.897 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:12.897 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:12.919 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-fuse 2026-02-23T10:27:13.044 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:05 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:13.108 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-mgr-rook-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:13.126 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:13.127 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:13.127 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:13.148 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-mgr-cephadm 2026-02-23T10:27:13.229 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:07 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:13.294 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:13.312 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:13.313 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:13.313 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:13.336 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-volume 2026-02-23T10:27:13.458 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:05 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:13.523 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-mgr-cephadm-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:13.541 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:13.541 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:13.542 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:13.563 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-osd 2026-02-23T10:27:13.647 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:07 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:13.711 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-volume-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:13.730 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:13.730 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:13.730 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:13.752 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados-devel 2026-02-23T10:27:13.876 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:05 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:13.942 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-osd-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:13.960 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:13.961 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:13.961 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:13.982 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-osd-classic 2026-02-23T10:27:14.059 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:08 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:14.124 INFO:teuthology.orchestra.run.trial038.stdout:Package librados-devel-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:14.142 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:14.143 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:14.143 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:14.165 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install libcephfs2 2026-02-23T10:27:14.294 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:06 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:14.359 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-osd-classic-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:14.378 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:14.379 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:14.379 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:14.401 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-fuse 2026-02-23T10:27:14.478 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:08 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:14.543 INFO:teuthology.orchestra.run.trial038.stdout:Package libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:14.561 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:14.562 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:14.562 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:14.585 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install libcephfs-devel 2026-02-23T10:27:14.707 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:06 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:14.772 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:14.790 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:14.790 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:14.791 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:14.811 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install ceph-volume 2026-02-23T10:27:14.896 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:08 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:14.961 INFO:teuthology.orchestra.run.trial038.stdout:Package libcephfs-devel-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:14.979 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:14.979 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:14.980 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:15.001 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados2 2026-02-23T10:27:15.121 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:07 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:15.185 INFO:teuthology.orchestra.run.trial153.stdout:Package ceph-volume-2:20.3.0-5439.ge371433f.el9.noarch is already installed. 2026-02-23T10:27:15.203 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:15.204 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:15.204 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:15.226 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install librados-devel 2026-02-23T10:27:15.318 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:09 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:15.384 INFO:teuthology.orchestra.run.trial038.stdout:Package librados2-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:15.402 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:15.402 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:15.403 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:15.425 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librbd1 2026-02-23T10:27:15.536 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:07 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:15.601 INFO:teuthology.orchestra.run.trial153.stdout:Package librados-devel-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:15.620 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:15.620 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:15.621 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:15.642 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install libcephfs2 2026-02-23T10:27:15.737 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:09 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:15.802 INFO:teuthology.orchestra.run.trial038.stdout:Package librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:15.821 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:15.821 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:15.821 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:15.843 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rados 2026-02-23T10:27:15.952 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:07 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:16.017 INFO:teuthology.orchestra.run.trial153.stdout:Package libcephfs2-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:16.035 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:16.036 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:16.036 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:16.057 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install libcephfs-devel 2026-02-23T10:27:16.156 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:10 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:16.220 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rados-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:16.238 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:16.239 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:16.239 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:16.260 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rgw 2026-02-23T10:27:16.368 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:08 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:16.434 INFO:teuthology.orchestra.run.trial153.stdout:Package libcephfs-devel-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:16.452 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:16.453 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:16.453 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:16.475 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install librados2 2026-02-23T10:27:16.570 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:10 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:16.634 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rgw-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:16.652 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:16.653 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:16.653 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:16.674 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-cephfs 2026-02-23T10:27:16.785 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:08 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:16.850 INFO:teuthology.orchestra.run.trial153.stdout:Package librados2-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:16.869 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:16.869 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:16.869 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:16.891 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install librbd1 2026-02-23T10:27:16.987 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:10 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:17.052 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-cephfs-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:17.070 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:17.071 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:17.071 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:17.092 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rbd 2026-02-23T10:27:17.205 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:09 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:17.271 INFO:teuthology.orchestra.run.trial153.stdout:Package librbd1-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:17.290 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:17.290 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:17.291 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:17.313 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install python3-rados 2026-02-23T10:27:17.401 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:11 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:17.465 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rbd-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:17.483 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:17.484 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:17.484 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:17.506 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-fuse 2026-02-23T10:27:17.624 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:09 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:17.688 INFO:teuthology.orchestra.run.trial153.stdout:Package python3-rados-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:17.707 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:17.707 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:17.708 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:17.729 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install python3-rgw 2026-02-23T10:27:17.821 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:11 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:17.886 INFO:teuthology.orchestra.run.trial038.stdout:Package rbd-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:17.905 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:17.905 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:17.905 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:17.927 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-mirror 2026-02-23T10:27:18.042 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:10 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:18.106 INFO:teuthology.orchestra.run.trial153.stdout:Package python3-rgw-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:18.124 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:18.125 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:18.125 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:18.147 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install python3-cephfs 2026-02-23T10:27:18.234 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:12 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:18.298 INFO:teuthology.orchestra.run.trial038.stdout:Package rbd-mirror-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:18.317 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:18.317 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:18.318 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:18.339 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-nbd 2026-02-23T10:27:18.457 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:10 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:18.523 INFO:teuthology.orchestra.run.trial153.stdout:Package python3-cephfs-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:18.541 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:18.542 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:18.542 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:18.564 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install python3-rbd 2026-02-23T10:27:18.651 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:12 ago on Mon 23 Feb 2026 10:27:06 AM UTC. 2026-02-23T10:27:18.716 INFO:teuthology.orchestra.run.trial038.stdout:Package rbd-nbd-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:18.734 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-23T10:27:18.735 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-23T10:27:18.735 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-23T10:27:18.759 DEBUG:teuthology.parallel:result is None 2026-02-23T10:27:18.877 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:10 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:18.941 INFO:teuthology.orchestra.run.trial153.stdout:Package python3-rbd-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:18.959 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:18.960 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:18.960 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:18.981 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install rbd-fuse 2026-02-23T10:27:19.294 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:11 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:19.360 INFO:teuthology.orchestra.run.trial153.stdout:Package rbd-fuse-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:19.379 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:19.379 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:19.379 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:19.402 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install rbd-mirror 2026-02-23T10:27:19.715 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:11 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:19.779 INFO:teuthology.orchestra.run.trial153.stdout:Package rbd-mirror-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:19.797 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:19.798 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:19.798 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:19.820 DEBUG:teuthology.orchestra.run.trial153:> sudo yum -y install rbd-nbd 2026-02-23T10:27:20.137 INFO:teuthology.orchestra.run.trial153.stdout:Last metadata expiration check: 0:00:12 ago on Mon 23 Feb 2026 10:27:08 AM UTC. 2026-02-23T10:27:20.201 INFO:teuthology.orchestra.run.trial153.stdout:Package rbd-nbd-2:20.3.0-5439.ge371433f.el9.x86_64 is already installed. 2026-02-23T10:27:20.220 INFO:teuthology.orchestra.run.trial153.stdout:Dependencies resolved. 2026-02-23T10:27:20.220 INFO:teuthology.orchestra.run.trial153.stdout:Nothing to do. 2026-02-23T10:27:20.221 INFO:teuthology.orchestra.run.trial153.stdout:Complete! 2026-02-23T10:27:20.247 DEBUG:teuthology.parallel:result is None 2026-02-23T10:27:20.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e371433fcd1390a0b82cfb18effc422c1f5d8c1d 2026-02-23T10:27:20.417 DEBUG:teuthology.orchestra.run.trial038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-23T10:27:20.439 INFO:teuthology.orchestra.run.trial038.stdout:20.3.0-5439.ge371433f.el9 2026-02-23T10:27:20.439 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5439.ge371433f.el9 2026-02-23T10:27:20.439 INFO:teuthology.task.install:The correct ceph version 20.3.0-5439.ge371433f is installed. 2026-02-23T10:27:20.442 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e371433fcd1390a0b82cfb18effc422c1f5d8c1d 2026-02-23T10:27:20.598 DEBUG:teuthology.orchestra.run.trial153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-23T10:27:20.619 INFO:teuthology.orchestra.run.trial153.stdout:20.3.0-5439.ge371433f.el9 2026-02-23T10:27:20.619 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5439.ge371433f.el9 2026-02-23T10:27:20.619 INFO:teuthology.task.install:The correct ceph version 20.3.0-5439.ge371433f is installed. 2026-02-23T10:27:20.622 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-23T10:27:20.623 INFO:teuthology.run_tasks:Running task ceph... 2026-02-23T10:27:20.719 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-23T10:27:20.720 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 777 /var/log/ceph 2026-02-23T10:27:20.723 DEBUG:teuthology.orchestra.run.trial153:> sudo chmod 777 /var/log/ceph 2026-02-23T10:27:20.747 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-23T10:27:20.748 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-23T10:27:20.791 DEBUG:teuthology.orchestra.run.trial153:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-23T10:27:20.816 INFO:tasks.ceph:Creating extra log directories... 2026-02-23T10:27:20.816 DEBUG:teuthology.orchestra.run.trial038:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-23T10:27:20.855 DEBUG:teuthology.orchestra.run.trial153:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-23T10:27:20.895 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-23T10:27:20.895 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug objclass': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-23T10:27:20.895 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial/66074', 'branch': 'main', 'description': 'crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/cls}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '66074', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug objclass': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d'}}, 'workunit': {'branch': 'wip-shraddhaag-enable-cephadm-tests', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d'}}, 'owner': 'scheduled_shraddhaag@soko04.front.sepia.ceph.com', 'priority': 49, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 66, 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'wip-shraddhaag-enable-cephadm-tests', 'suite_path': '/home/teuthworker/src/github.com_shraddhaag_ceph_e371433fcd1390a0b82cfb18effc422c1f5d8c1d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/shraddhaag/ceph.git', 'suite_sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d', 'targets': {'trial038.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtfCwuckGkN6i1n7kv7PxQkFv2itGL13cIm1t1CIrPT/6vdzmqLsPE67bnoxeG/1ALqD4uTtQguyHDSV9NMNrY=', 'trial153.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJogg+T3xVpsEzYIoOYY8NxCM23gi/MYPU21fpKfPtAQACeva3HTLKbRtdu0NMj5PBkCk1PA9X1dVlfDopSaMRk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug objclass': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_cmpomap.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-23_09:58:48', 'tube': 'trial', 'user': 'shraddhaag', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653'} 2026-02-23T10:27:20.896 DEBUG:teuthology.orchestra.run.trial038:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-23T10:27:20.920 DEBUG:teuthology.orchestra.run.trial153:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-23T10:27:20.937 DEBUG:teuthology.orchestra.run.trial038:> sudo install -d -m0777 -- /var/run/ceph 2026-02-23T10:27:20.976 DEBUG:teuthology.orchestra.run.trial153:> sudo install -d -m0777 -- /var/run/ceph 2026-02-23T10:27:21.003 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:27:21.003 DEBUG:teuthology.orchestra.run.trial038:> dd if=/scratch_devs of=/dev/stdout 2026-02-23T10:27:21.055 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-23T10:27:21.056 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_1 2026-02-23T10:27:21.113 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-23T10:27:21.113 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-23T10:27:21.113 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 917 Links: 1 2026-02-23T10:27:21.113 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-23T10:27:21.114 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-23T10:27:21.114 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-23 10:24:22.755749249 +0000 2026-02-23T10:27:21.114 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-23 10:20:28.796728932 +0000 2026-02-23T10:27:21.114 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-23 10:20:28.796728932 +0000 2026-02-23T10:27:21.114 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-23 10:20:28.796728932 +0000 2026-02-23T10:27:21.114 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-23T10:27:21.177 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-23T10:27:21.177 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-23T10:27:21.177 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000153955 s, 3.3 MB/s 2026-02-23T10:27:21.178 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-23T10:27:21.236 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_2 2026-02-23T10:27:21.293 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-23T10:27:21.293 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-23T10:27:21.294 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 945 Links: 1 2026-02-23T10:27:21.294 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-23T10:27:21.294 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-23T10:27:21.294 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-23 10:24:22.755749249 +0000 2026-02-23T10:27:21.294 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-23 10:20:29.039719500 +0000 2026-02-23T10:27:21.294 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-23 10:20:29.039719500 +0000 2026-02-23T10:27:21.294 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-23 10:20:29.039719500 +0000 2026-02-23T10:27:21.294 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-23T10:27:21.357 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-23T10:27:21.357 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-23T10:27:21.357 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000152322 s, 3.4 MB/s 2026-02-23T10:27:21.359 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-23T10:27:21.417 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_3 2026-02-23T10:27:21.473 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-23T10:27:21.474 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-23T10:27:21.474 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 971 Links: 1 2026-02-23T10:27:21.474 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-23T10:27:21.474 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-23T10:27:21.474 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-23 10:24:22.755749249 +0000 2026-02-23T10:27:21.474 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-23 10:20:29.286709912 +0000 2026-02-23T10:27:21.474 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-23 10:20:29.286709912 +0000 2026-02-23T10:27:21.474 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-23 10:20:29.286709912 +0000 2026-02-23T10:27:21.475 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-23T10:27:21.536 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-23T10:27:21.536 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-23T10:27:21.536 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000154978 s, 3.3 MB/s 2026-02-23T10:27:21.538 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-23T10:27:21.595 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_4 2026-02-23T10:27:21.652 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-23T10:27:21.653 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-23T10:27:21.653 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 1000 Links: 1 2026-02-23T10:27:21.653 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-23T10:27:21.653 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-23T10:27:21.653 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-23 10:24:22.756749225 +0000 2026-02-23T10:27:21.653 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-23 10:20:29.519700868 +0000 2026-02-23T10:27:21.653 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-23 10:20:29.519700868 +0000 2026-02-23T10:27:21.653 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-23 10:20:29.519700868 +0000 2026-02-23T10:27:21.654 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-23T10:27:21.717 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-23T10:27:21.718 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-23T10:27:21.718 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000141732 s, 3.6 MB/s 2026-02-23T10:27:21.719 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-23T10:27:21.776 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-23T10:27:21.776 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:27:21.776 DEBUG:teuthology.orchestra.run.trial153:> dd if=/scratch_devs of=/dev/stdout 2026-02-23T10:27:21.794 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-23T10:27:21.795 DEBUG:teuthology.orchestra.run.trial153:> stat /dev/vg_nvme/lv_1 2026-02-23T10:27:21.851 INFO:teuthology.orchestra.run.trial153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-23T10:27:21.851 INFO:teuthology.orchestra.run.trial153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-23T10:27:21.851 INFO:teuthology.orchestra.run.trial153.stdout:Device: 6h/6d Inode: 915 Links: 1 2026-02-23T10:27:21.851 INFO:teuthology.orchestra.run.trial153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-23T10:27:21.851 INFO:teuthology.orchestra.run.trial153.stdout:Context: system_u:object_r:device_t:s0 2026-02-23T10:27:21.851 INFO:teuthology.orchestra.run.trial153.stdout:Access: 2026-02-23 10:24:47.400363417 +0000 2026-02-23T10:27:21.851 INFO:teuthology.orchestra.run.trial153.stdout:Modify: 2026-02-23 10:20:29.683679556 +0000 2026-02-23T10:27:21.851 INFO:teuthology.orchestra.run.trial153.stdout:Change: 2026-02-23 10:20:29.683679556 +0000 2026-02-23T10:27:21.852 INFO:teuthology.orchestra.run.trial153.stdout: Birth: 2026-02-23 10:20:29.683679556 +0000 2026-02-23T10:27:21.852 DEBUG:teuthology.orchestra.run.trial153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-23T10:27:21.913 INFO:teuthology.orchestra.run.trial153.stderr:1+0 records in 2026-02-23T10:27:21.913 INFO:teuthology.orchestra.run.trial153.stderr:1+0 records out 2026-02-23T10:27:21.914 INFO:teuthology.orchestra.run.trial153.stderr:512 bytes copied, 0.000159238 s, 3.2 MB/s 2026-02-23T10:27:21.915 DEBUG:teuthology.orchestra.run.trial153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-23T10:27:21.973 DEBUG:teuthology.orchestra.run.trial153:> stat /dev/vg_nvme/lv_2 2026-02-23T10:27:22.028 INFO:teuthology.orchestra.run.trial153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-23T10:27:22.028 INFO:teuthology.orchestra.run.trial153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-23T10:27:22.028 INFO:teuthology.orchestra.run.trial153.stdout:Device: 6h/6d Inode: 943 Links: 1 2026-02-23T10:27:22.029 INFO:teuthology.orchestra.run.trial153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-23T10:27:22.029 INFO:teuthology.orchestra.run.trial153.stdout:Context: system_u:object_r:device_t:s0 2026-02-23T10:27:22.029 INFO:teuthology.orchestra.run.trial153.stdout:Access: 2026-02-23 10:24:47.401363413 +0000 2026-02-23T10:27:22.029 INFO:teuthology.orchestra.run.trial153.stdout:Modify: 2026-02-23 10:20:29.924682821 +0000 2026-02-23T10:27:22.029 INFO:teuthology.orchestra.run.trial153.stdout:Change: 2026-02-23 10:20:29.924682821 +0000 2026-02-23T10:27:22.029 INFO:teuthology.orchestra.run.trial153.stdout: Birth: 2026-02-23 10:20:29.924682821 +0000 2026-02-23T10:27:22.029 DEBUG:teuthology.orchestra.run.trial153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-23T10:27:22.091 INFO:teuthology.orchestra.run.trial153.stderr:1+0 records in 2026-02-23T10:27:22.091 INFO:teuthology.orchestra.run.trial153.stderr:1+0 records out 2026-02-23T10:27:22.091 INFO:teuthology.orchestra.run.trial153.stderr:512 bytes copied, 0.000153828 s, 3.3 MB/s 2026-02-23T10:27:22.092 DEBUG:teuthology.orchestra.run.trial153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-23T10:27:22.149 DEBUG:teuthology.orchestra.run.trial153:> stat /dev/vg_nvme/lv_3 2026-02-23T10:27:22.205 INFO:teuthology.orchestra.run.trial153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-23T10:27:22.206 INFO:teuthology.orchestra.run.trial153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-23T10:27:22.206 INFO:teuthology.orchestra.run.trial153.stdout:Device: 6h/6d Inode: 971 Links: 1 2026-02-23T10:27:22.206 INFO:teuthology.orchestra.run.trial153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-23T10:27:22.206 INFO:teuthology.orchestra.run.trial153.stdout:Context: system_u:object_r:device_t:s0 2026-02-23T10:27:22.206 INFO:teuthology.orchestra.run.trial153.stdout:Access: 2026-02-23 10:24:47.401363413 +0000 2026-02-23T10:27:22.206 INFO:teuthology.orchestra.run.trial153.stdout:Modify: 2026-02-23 10:20:30.173686194 +0000 2026-02-23T10:27:22.206 INFO:teuthology.orchestra.run.trial153.stdout:Change: 2026-02-23 10:20:30.173686194 +0000 2026-02-23T10:27:22.206 INFO:teuthology.orchestra.run.trial153.stdout: Birth: 2026-02-23 10:20:30.173686194 +0000 2026-02-23T10:27:22.207 DEBUG:teuthology.orchestra.run.trial153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-23T10:27:22.267 INFO:teuthology.orchestra.run.trial153.stderr:1+0 records in 2026-02-23T10:27:22.267 INFO:teuthology.orchestra.run.trial153.stderr:1+0 records out 2026-02-23T10:27:22.267 INFO:teuthology.orchestra.run.trial153.stderr:512 bytes copied, 0.000149039 s, 3.4 MB/s 2026-02-23T10:27:22.269 DEBUG:teuthology.orchestra.run.trial153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-23T10:27:22.325 DEBUG:teuthology.orchestra.run.trial153:> stat /dev/vg_nvme/lv_4 2026-02-23T10:27:22.381 INFO:teuthology.orchestra.run.trial153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-23T10:27:22.382 INFO:teuthology.orchestra.run.trial153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-23T10:27:22.382 INFO:teuthology.orchestra.run.trial153.stdout:Device: 6h/6d Inode: 1000 Links: 1 2026-02-23T10:27:22.382 INFO:teuthology.orchestra.run.trial153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-23T10:27:22.382 INFO:teuthology.orchestra.run.trial153.stdout:Context: system_u:object_r:device_t:s0 2026-02-23T10:27:22.382 INFO:teuthology.orchestra.run.trial153.stdout:Access: 2026-02-23 10:24:47.401363413 +0000 2026-02-23T10:27:22.382 INFO:teuthology.orchestra.run.trial153.stdout:Modify: 2026-02-23 10:20:30.424689594 +0000 2026-02-23T10:27:22.382 INFO:teuthology.orchestra.run.trial153.stdout:Change: 2026-02-23 10:20:30.424689594 +0000 2026-02-23T10:27:22.382 INFO:teuthology.orchestra.run.trial153.stdout: Birth: 2026-02-23 10:20:30.424689594 +0000 2026-02-23T10:27:22.383 DEBUG:teuthology.orchestra.run.trial153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-23T10:27:22.444 INFO:teuthology.orchestra.run.trial153.stderr:1+0 records in 2026-02-23T10:27:22.444 INFO:teuthology.orchestra.run.trial153.stderr:1+0 records out 2026-02-23T10:27:22.444 INFO:teuthology.orchestra.run.trial153.stderr:512 bytes copied, 0.000146434 s, 3.5 MB/s 2026-02-23T10:27:22.445 DEBUG:teuthology.orchestra.run.trial153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-23T10:27:22.503 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-23T10:27:22.503 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial153.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-23T10:27:22.503 INFO:tasks.ceph:Generating config... 2026-02-23T10:27:22.505 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-23T10:27:22.505 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-23T10:27:22.505 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-23T10:27:22.505 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-23T10:27:22.506 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-23T10:27:22.506 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-23T10:27:22.506 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-23T10:27:22.506 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-23T10:27:22.506 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-23T10:27:22.506 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-23T10:27:22.506 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-23T10:27:22.506 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-23T10:27:22.506 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-23T10:27:22.506 INFO:tasks.ceph:[osd] debug objclass = 20 2026-02-23T10:27:22.506 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-23T10:27:22.506 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-23T10:27:22.507 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-23T10:27:22.508 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-23T10:27:22.508 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-23T10:27:22.508 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-23T10:27:22.508 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-23T10:27:22.508 INFO:tasks.ceph:[osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-23T10:27:22.508 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-23T10:27:22.508 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-23T10:27:22.508 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-23T10:27:22.508 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-23T10:27:22.508 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-23T10:27:22.508 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-23T10:27:22.508 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-23T10:27:22.509 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-23T10:27:22.509 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-23T10:27:22.509 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-23T10:27:22.509 INFO:tasks.ceph:Setting up mon.a... 2026-02-23T10:27:22.509 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-23T10:27:22.552 INFO:teuthology.orchestra.run.trial038.stdout:creating /etc/ceph/ceph.keyring 2026-02-23T10:27:22.555 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-23T10:27:22.638 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-23T10:27:22.702 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.38')] 2026-02-23T10:27:22.703 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.38', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug objclass': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-23T10:27:22.704 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:27:22.704 DEBUG:teuthology.orchestra.run.trial038:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-23T10:27:22.758 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.38 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-23T10:27:22.837 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:22.835+0000 7f60ef087f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:22.837 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:22.835+0000 7f60ef087f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:22.837 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:22.835+0000 7f60ef087f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:22.838 INFO:teuthology.orchestra.run.trial038.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-23T10:27:22.839 INFO:teuthology.orchestra.run.trial038.stdout:monmaptool: generated fsid ce093877-a685-4ffe-9065-5da136907d50 2026-02-23T10:27:22.839 INFO:teuthology.orchestra.run.trial038.stdout:setting min_mon_release = tentacle 2026-02-23T10:27:22.839 INFO:teuthology.orchestra.run.trial038.stdout:epoch 0 2026-02-23T10:27:22.839 INFO:teuthology.orchestra.run.trial038.stdout:fsid ce093877-a685-4ffe-9065-5da136907d50 2026-02-23T10:27:22.839 INFO:teuthology.orchestra.run.trial038.stdout:last_changed 2026-02-23T10:27:22.836605+0000 2026-02-23T10:27:22.839 INFO:teuthology.orchestra.run.trial038.stdout:created 2026-02-23T10:27:22.836605+0000 2026-02-23T10:27:22.839 INFO:teuthology.orchestra.run.trial038.stdout:min_mon_release 20 (tentacle) 2026-02-23T10:27:22.839 INFO:teuthology.orchestra.run.trial038.stdout:election_strategy: 1 2026-02-23T10:27:22.839 INFO:teuthology.orchestra.run.trial038.stdout:0: [v2:10.20.193.38:3300/0,v1:10.20.193.38:6789/0] mon.a 2026-02-23T10:27:22.839 INFO:teuthology.orchestra.run.trial038.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-23T10:27:22.840 DEBUG:teuthology.orchestra.run.trial038:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-23T10:27:22.896 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ce093877-a685-4ffe-9065-5da136907d50... 2026-02-23T10:27:22.898 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-23T10:27:22.939 DEBUG:teuthology.orchestra.run.trial153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-23T10:27:22.979 INFO:teuthology.orchestra.run.trial038.stdout:[global] 2026-02-23T10:27:22.979 INFO:teuthology.orchestra.run.trial038.stdout: chdir = "" 2026-02-23T10:27:22.979 INFO:teuthology.orchestra.run.trial038.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-23T10:27:22.979 INFO:teuthology.orchestra.run.trial038.stdout: auth supported = cephx 2026-02-23T10:27:22.979 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: filestore xattr use omap = true 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: mon clock drift allowed = 1.000 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: osd crush chooseleaf type = 0 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: auth debug = true 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: ms die on old message = true 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: ms die on bug = true 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-23T10:27:22.980 INFO:teuthology.orchestra.run.trial038.stdout: mon pg warn max object skew = 0 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: # disable pg_autoscaler by default for new pools 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: osd pool default size = 2 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: mon osd allow primary affinity = true 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: mon osd allow pg remap = true 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on legacy crush tunables = false 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on crush straw calc version zero = false 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on no sortbitwise = false 2026-02-23T10:27:22.981 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on osd down out interval zero = false 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on too few osds = false 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: mon_allow_pool_size_one = true 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: osd default data pool replay window = 5 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: mon allow pool delete = true 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.982 INFO:teuthology.orchestra.run.trial038.stdout: mon cluster log file level = debug 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: debug asserts on shutdown = true 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: mon health detail to clog = false 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: mon host = 10.20.193.38 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: ms client mode = crc 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: ms cluster mode = crc 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: ms mon client mode = crc 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: ms mon cluster mode = crc 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: ms mon service mode = crc 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: ms service mode = crc 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: fsid = ce093877-a685-4ffe-9065-5da136907d50 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.983 INFO:teuthology.orchestra.run.trial038.stdout:[osd] 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: osd journal size = 100 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: osd scrub load threshold = 5.0 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: osd scrub max interval = 600 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: osd mclock profile = high_recovery_ops 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: osd mclock skip benchmark = true 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: osd recover clone overlap = true 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: osd recovery max chunk = 1048576 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: osd debug shutdown = true 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: osd debug op order = true 2026-02-23T10:27:22.984 INFO:teuthology.orchestra.run.trial038.stdout: osd debug verify stray on activate = true 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: osd debug trim objects = true 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: osd open classes on start = true 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: osd debug pg log writeout = true 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: osd deep scrub update digest min age = 30 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: osd map max advance = 10 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: journal zero on create = true 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.985 INFO:teuthology.orchestra.run.trial038.stdout: filestore ondisk finisher threads = 3 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: filestore apply finisher threads = 3 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: bdev debug aio = true 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: osd debug misdirected ops = true 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: debug monc = 20 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: crimson cpu num = 4 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: crimson osd obc lru size = 10 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: debug ms = 20 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: debug objclass = 20 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: debug osd = 20 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore = 20 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore backref = 20 2026-02-23T10:27:22.986 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore cache = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore cleaner = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore device = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore epm = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore fixedkv tree = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore journal = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore lba = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore odata = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore omap = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore onode = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore t = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore tm = 20 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: osd objectstore = seastore 2026-02-23T10:27:22.987 INFO:teuthology.orchestra.run.trial038.stdout: osd shutdown pgref assert = True 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: seastore cachepin size pershard = 64M 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: seastore cachepin type = 2Q 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: seastore max concurrent transactions = 8 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout:[mgr] 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: debug ms = 1 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: debug mgr = 20 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: debug mon = 20 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: debug auth = 20 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min pgs per osd = 4 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min bytes per osd = 10 2026-02-23T10:27:22.988 INFO:teuthology.orchestra.run.trial038.stdout: mgr/telemetry/nag = false 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout:[mon] 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: debug ms = 1 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: debug mon = 20 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: debug paxos = 20 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: debug auth = 20 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: mon data avail warn = 5 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: mon mgr mkfs grace = 240 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min pgs per osd = 4 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: mon osd reporter subtree level = osd 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: mon osd prime pg temp = true 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min bytes per osd = 10 2026-02-23T10:27:22.989 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: auth mon ticket ttl = 660 # 11m 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: auth service ticket ttl = 240 # 4m 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: # don't complain about insecure global_id in the test suite 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: # 1m isn't quite enough 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: mon_down_mkfs_grace = 2m 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_filestore_osds = false 2026-02-23T10:27:22.990 INFO:teuthology.orchestra.run.trial038.stdout: mon min osdmap epochs = 50 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout: mon osdmap full prune interval = 2 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout: mon osdmap full prune min = 15 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout: mon osdmap full prune txsize = 2 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout: paxos service trim min = 10 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout: osd pool default crimson = True 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout:[client] 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout: rgw cache enabled = true 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout: rgw enable ops log = true 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout: rgw enable usage log = true 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-23T10:27:22.991 INFO:teuthology.orchestra.run.trial038.stdout:[mon.a] 2026-02-23T10:27:22.993 INFO:teuthology.orchestra.run.trial153.stdout:[global] 2026-02-23T10:27:22.994 INFO:teuthology.orchestra.run.trial153.stdout: chdir = "" 2026-02-23T10:27:22.994 INFO:teuthology.orchestra.run.trial153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-23T10:27:22.994 INFO:teuthology.orchestra.run.trial153.stdout: auth supported = cephx 2026-02-23T10:27:22.994 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.994 INFO:teuthology.orchestra.run.trial153.stdout: filestore xattr use omap = true 2026-02-23T10:27:22.994 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.994 INFO:teuthology.orchestra.run.trial153.stdout: mon clock drift allowed = 1.000 2026-02-23T10:27:22.994 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.994 INFO:teuthology.orchestra.run.trial153.stdout: osd crush chooseleaf type = 0 2026-02-23T10:27:22.994 INFO:teuthology.orchestra.run.trial153.stdout: auth debug = true 2026-02-23T10:27:22.994 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: ms die on old message = true 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: ms die on bug = true 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: mon pg warn max object skew = 0 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: # disable pg_autoscaler by default for new pools 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: osd pool default size = 2 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.995 INFO:teuthology.orchestra.run.trial153.stdout: mon osd allow primary affinity = true 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: mon osd allow pg remap = true 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: mon warn on legacy crush tunables = false 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: mon warn on crush straw calc version zero = false 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: mon warn on no sortbitwise = false 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: mon warn on osd down out interval zero = false 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: mon warn on too few osds = false 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: mon_allow_pool_size_one = true 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.996 INFO:teuthology.orchestra.run.trial153.stdout: osd default data pool replay window = 5 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: mon allow pool delete = true 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: mon cluster log file level = debug 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: debug asserts on shutdown = true 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: mon health detail to clog = false 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: mon host = 10.20.193.38 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: ms client mode = crc 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: ms cluster mode = crc 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: ms mon client mode = crc 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: ms mon cluster mode = crc 2026-02-23T10:27:22.997 INFO:teuthology.orchestra.run.trial153.stdout: ms mon service mode = crc 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout: ms service mode = crc 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout: fsid = ce093877-a685-4ffe-9065-5da136907d50 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout:[osd] 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout: osd journal size = 100 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout: osd scrub load threshold = 5.0 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout: osd scrub max interval = 600 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout: osd mclock profile = high_recovery_ops 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout: osd mclock skip benchmark = true 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.998 INFO:teuthology.orchestra.run.trial153.stdout: osd recover clone overlap = true 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: osd recovery max chunk = 1048576 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: osd debug shutdown = true 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: osd debug op order = true 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: osd debug verify stray on activate = true 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: osd debug trim objects = true 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: osd open classes on start = true 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: osd debug pg log writeout = true 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: osd deep scrub update digest min age = 30 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:22.999 INFO:teuthology.orchestra.run.trial153.stdout: osd map max advance = 10 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: journal zero on create = true 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: filestore ondisk finisher threads = 3 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: filestore apply finisher threads = 3 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: bdev debug aio = true 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: osd debug misdirected ops = true 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: debug monc = 20 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: crimson cpu num = 4 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: crimson osd obc lru size = 10 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: debug ms = 20 2026-02-23T10:27:23.000 INFO:teuthology.orchestra.run.trial153.stdout: debug objclass = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug osd = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore backref = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore cache = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore cleaner = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore device = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore epm = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore fixedkv tree = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore journal = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore lba = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore odata = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore omap = 20 2026-02-23T10:27:23.001 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore onode = 20 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore t = 20 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: debug seastore tm = 20 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: osd objectstore = seastore 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: osd shutdown pgref assert = True 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: seastore cachepin size pershard = 64M 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: seastore cachepin type = 2Q 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: seastore max concurrent transactions = 8 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout:[mgr] 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: debug ms = 1 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: debug mgr = 20 2026-02-23T10:27:23.002 INFO:teuthology.orchestra.run.trial153.stdout: debug mon = 20 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: debug auth = 20 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: mon reweight min pgs per osd = 4 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: mon reweight min bytes per osd = 10 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: mgr/telemetry/nag = false 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout:[mon] 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: debug ms = 1 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: debug mon = 20 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: debug paxos = 20 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: debug auth = 20 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: mon data avail warn = 5 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: mon mgr mkfs grace = 240 2026-02-23T10:27:23.003 INFO:teuthology.orchestra.run.trial153.stdout: mon reweight min pgs per osd = 4 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: mon osd reporter subtree level = osd 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: mon osd prime pg temp = true 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: mon reweight min bytes per osd = 10 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: auth mon ticket ttl = 660 # 11m 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: auth service ticket ttl = 240 # 4m 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: # don't complain about insecure global_id in the test suite 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-23T10:27:23.004 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: # 1m isn't quite enough 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: mon_down_mkfs_grace = 2m 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: mon_warn_on_filestore_osds = false 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: mon min osdmap epochs = 50 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: mon osdmap full prune interval = 2 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: mon osdmap full prune min = 15 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: mon osdmap full prune txsize = 2 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: paxos service trim min = 10 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: osd pool default crimson = True 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout:[client] 2026-02-23T10:27:23.005 INFO:teuthology.orchestra.run.trial153.stdout: rgw cache enabled = true 2026-02-23T10:27:23.006 INFO:teuthology.orchestra.run.trial153.stdout: rgw enable ops log = true 2026-02-23T10:27:23.006 INFO:teuthology.orchestra.run.trial153.stdout: rgw enable usage log = true 2026-02-23T10:27:23.006 INFO:teuthology.orchestra.run.trial153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-23T10:27:23.006 INFO:teuthology.orchestra.run.trial153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-23T10:27:23.006 INFO:teuthology.orchestra.run.trial153.stdout:[mon.a] 2026-02-23T10:27:23.006 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-23T10:27:23.006 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-23T10:27:23.084 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-23T10:27:23.084 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:27:23.084 DEBUG:teuthology.orchestra.run.trial038:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-23T10:27:23.101 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:27:23.101 DEBUG:teuthology.orchestra.run.trial038:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-23T10:27:23.157 INFO:tasks.ceph:Sending monmap to node ubuntu@trial038.front.sepia.ceph.com 2026-02-23T10:27:23.157 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:27:23.157 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-23T10:27:23.157 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-23T10:27:23.229 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:27:23.230 DEBUG:teuthology.orchestra.run.trial038:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-23T10:27:23.287 INFO:tasks.ceph:Sending monmap to node ubuntu@trial153.front.sepia.ceph.com 2026-02-23T10:27:23.287 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:27:23.287 DEBUG:teuthology.orchestra.run.trial153:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-23T10:27:23.287 DEBUG:teuthology.orchestra.run.trial153:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-23T10:27:23.322 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:27:23.323 DEBUG:teuthology.orchestra.run.trial153:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-23T10:27:23.378 INFO:tasks.ceph:Setting up mon nodes... 2026-02-23T10:27:23.379 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-23T10:27:23.379 DEBUG:teuthology.orchestra.run.trial153:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-23T10:27:23.470 INFO:teuthology.orchestra.run.trial153.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-23T10:27:23.473 INFO:tasks.ceph:Setting up mds nodes... 2026-02-23T10:27:23.473 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-23T10:27:23.474 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-23T10:27:23.516 INFO:teuthology.orchestra.run.trial038.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-23T10:27:23.528 DEBUG:teuthology.orchestra.run.trial153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2026-02-23T10:27:23.568 INFO:teuthology.orchestra.run.trial153.stdout:creating /etc/ceph/ceph.client.1.keyring 2026-02-23T10:27:23.579 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-23T10:27:23.579 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial153.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-23T10:27:23.580 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-23T10:27:23.603 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-23T10:27:23.604 INFO:tasks.ceph:role: osd.0 2026-02-23T10:27:23.604 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial038.front.sepia.ceph.com 2026-02-23T10:27:23.604 DEBUG:teuthology.orchestra.run.trial038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-23T10:27:23.670 INFO:teuthology.orchestra.run.trial038.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-23T10:27:23.671 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-23T10:27:23.671 INFO:teuthology.orchestra.run.trial038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-23T10:27:23.671 INFO:teuthology.orchestra.run.trial038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-23T10:27:23.671 INFO:teuthology.orchestra.run.trial038.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-23T10:27:23.671 INFO:teuthology.orchestra.run.trial038.stdout: = sunit=0 swidth=0 blks 2026-02-23T10:27:23.671 INFO:teuthology.orchestra.run.trial038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-23T10:27:23.671 INFO:teuthology.orchestra.run.trial038.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-23T10:27:23.671 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-23T10:27:23.671 INFO:teuthology.orchestra.run.trial038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-23T10:27:24.291 INFO:teuthology.orchestra.run.trial038.stdout:Discarding blocks...Done. 2026-02-23T10:27:24.293 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial038.front.sepia.ceph.com -o noatime 2026-02-23T10:27:24.293 DEBUG:teuthology.orchestra.run.trial038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-23T10:27:24.362 DEBUG:teuthology.orchestra.run.trial038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-23T10:27:24.429 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-23T10:27:24.494 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-23T10:27:24.494 INFO:tasks.ceph:role: osd.1 2026-02-23T10:27:24.494 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial038.front.sepia.ceph.com 2026-02-23T10:27:24.494 DEBUG:teuthology.orchestra.run.trial038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-23T10:27:24.559 INFO:teuthology.orchestra.run.trial038.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-23T10:27:24.559 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-23T10:27:24.559 INFO:teuthology.orchestra.run.trial038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-23T10:27:24.559 INFO:teuthology.orchestra.run.trial038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-23T10:27:24.559 INFO:teuthology.orchestra.run.trial038.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-23T10:27:24.559 INFO:teuthology.orchestra.run.trial038.stdout: = sunit=0 swidth=0 blks 2026-02-23T10:27:24.559 INFO:teuthology.orchestra.run.trial038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-23T10:27:24.560 INFO:teuthology.orchestra.run.trial038.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-23T10:27:24.560 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-23T10:27:24.560 INFO:teuthology.orchestra.run.trial038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-23T10:27:25.114 INFO:teuthology.orchestra.run.trial038.stdout:Discarding blocks...Done. 2026-02-23T10:27:25.116 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial038.front.sepia.ceph.com -o noatime 2026-02-23T10:27:25.116 DEBUG:teuthology.orchestra.run.trial038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-23T10:27:25.184 DEBUG:teuthology.orchestra.run.trial038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-23T10:27:25.251 DEBUG:teuthology.orchestra.run.trial038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-23T10:27:25.404 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: parsing early config 2026-02-23T10:27:25.458 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:25,457 seastar - Reactor backend: linux-aio 2026-02-23T10:27:25.475 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:25,474 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:25.476 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:25,475 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:25.478 INFO:teuthology.orchestra.run.trial038.stderr:==38572==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-23T10:27:25.533 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:25,532 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-23T10:27:25.533 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:25,532 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-23T10:27:25.534 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:25,532 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-23T10:27:25.534 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:25,532 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-23T10:27:25.534 INFO:teuthology.orchestra.run.trial038.stderr:DEBUG 2026-02-23 10:27:25,532 [shard 0:main] seastar - reactor::drain 2026-02-23T10:27:27.586 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-23T10:27:27.586 INFO:teuthology.orchestra.run.trial038.stderr:Suppressions used: 2026-02-23T10:27:27.586 INFO:teuthology.orchestra.run.trial038.stderr: count bytes template 2026-02-23T10:27:27.586 INFO:teuthology.orchestra.run.trial038.stderr: 1 8 MallocExtension::Register 2026-02-23T10:27:27.586 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-23T10:27:27.586 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-23T10:27:27.605 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-23T10:27:27.605 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-23T10:27:27.650 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-23T10:27:27.650 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-23T10:27:27.657 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:27,656 seastar - Reactor backend: linux-aio 2026-02-23T10:27:27.668 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:27,667 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:27.668 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:27,667 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:27.672 INFO:teuthology.orchestra.run.trial038.stderr:==38570==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-23T10:27:27.672 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:27,671 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-23T10:27:27.731 INFO:teuthology.orchestra.run.trial038.stderr:DEBUG 2026-02-23 10:27:27,730 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-23T10:27:27.809 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x16507 0x9c95ab6 0x9c95ee6 0x9ca0178 0x9ca45f1 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:27.810 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129784 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0xc08522 0x7ce545b 0x7ce55fa 0x7bd8faf 0x7be8413 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:27.810 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfcaf3 /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x157c7 0x1091317 0xf2e3ce 0x9b7ddd8 0x9b7e0d5 0x9b7e348 0x9b7e5fe 0x9b7ebc0 0x9b83ce5 0x9b843c5 0x9c9ff46 0x9ca45f1 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:27.962 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x316ca9f 0x9b7e336 0x9b8006f 0x9c95c77 0x9c95ee6 0x9ca0178 0x9ca45f1 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:27.962 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f 0x9b7e51b 0x9b7f0ad 0x9b7f5ab 0x9b7fbdb 0x9b7fd45 0x9c8fd63 0x9ca2b6a 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:27.962 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x1125e3c 0xf2e3c3 0x9b7dc0b 0x9b7e0d5 0x9b7e348 0x9b8006f 0x9b805bc 0x9b807bb 0x9c9578b 0x9c95ee6 0x9ca0178 0x9ca45f1 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:30.079 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-23T10:27:30.079 INFO:teuthology.orchestra.run.trial038.stderr:Suppressions used: 2026-02-23T10:27:30.079 INFO:teuthology.orchestra.run.trial038.stderr: count bytes template 2026-02-23T10:27:30.079 INFO:teuthology.orchestra.run.trial038.stderr: 1 8 MallocExtension::Register 2026-02-23T10:27:30.079 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-23T10:27:30.080 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-23T10:27:30.111 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-23T10:27:30.136 DEBUG:teuthology.orchestra.run.trial038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-23T10:27:30.290 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: parsing early config 2026-02-23T10:27:30.343 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:30,342 seastar - Reactor backend: linux-aio 2026-02-23T10:27:30.358 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:30,357 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:30.358 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:30,357 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:30.360 INFO:teuthology.orchestra.run.trial038.stderr:==38638==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-23T10:27:30.415 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:30,414 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-23T10:27:30.415 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:30,414 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-23T10:27:30.416 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:30,414 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-23T10:27:30.416 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:30,414 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-23T10:27:30.416 INFO:teuthology.orchestra.run.trial038.stderr:DEBUG 2026-02-23 10:27:30,414 [shard 0:main] seastar - reactor::drain 2026-02-23T10:27:32.432 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-23T10:27:32.432 INFO:teuthology.orchestra.run.trial038.stderr:Suppressions used: 2026-02-23T10:27:32.433 INFO:teuthology.orchestra.run.trial038.stderr: count bytes template 2026-02-23T10:27:32.433 INFO:teuthology.orchestra.run.trial038.stderr: 1 8 MallocExtension::Register 2026-02-23T10:27:32.433 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-23T10:27:32.433 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-23T10:27:32.452 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-23T10:27:32.452 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-23T10:27:32.493 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-23T10:27:32.493 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-23T10:27:32.500 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:32,499 seastar - Reactor backend: linux-aio 2026-02-23T10:27:32.512 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:32,511 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:32.513 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:32,512 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:32.517 INFO:teuthology.orchestra.run.trial038.stderr:==38636==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-23T10:27:32.517 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-23 10:27:32,516 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-23T10:27:32.576 INFO:teuthology.orchestra.run.trial038.stderr:DEBUG 2026-02-23 10:27:32,575 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-23T10:27:32.655 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f 0x9b03d92 0x9c864c2 0x9c865bb 0x7be87ab 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:32.655 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f 0x9b7e51b 0x9b7f0ad 0x9b7f5ab 0x9b7fbdb 0x9b7fd45 0x9c8fd63 0x9ca2b6a 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:32.655 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b36a /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xb69729 0x10912a0 0xf2e3ce 0x9b7dc0b 0x9b7e0d5 0x9b7e348 0x9b8006f 0x9b805bc 0x9b807bb 0x9c9578b 0x9c95ee6 0x9ca0178 0x9ca45f1 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:32.802 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfd253 /lib64/libasan.so.8+0xedcb5 0xe0b7c9 0xe0b9ff 0x7be8003 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:32.802 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f 0x1091260 0xf2e3ce 0x9b7dc0b 0x9b7e0d5 0x9b7e348 0x9b7e5fe 0x9b7f0ad 0x9b7f5ab 0x9b7fbdb 0x9b7fd45 0x9c8fd63 0x9ca2b6a 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:32.802 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ed7 /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x3606bbd 0x9b8b6b3 0x9b8b796 0x9c872e7 0x9c8747c 0x9c8780e 0x9c87a19 0x9c87d55 0x9c88176 0x9c9f909 0x9ca45f1 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:34.946 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-23T10:27:34.946 INFO:teuthology.orchestra.run.trial038.stderr:Suppressions used: 2026-02-23T10:27:34.946 INFO:teuthology.orchestra.run.trial038.stderr: count bytes template 2026-02-23T10:27:34.947 INFO:teuthology.orchestra.run.trial038.stderr: 1 8 MallocExtension::Register 2026-02-23T10:27:34.947 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-23T10:27:34.947 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-23T10:27:34.978 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-23T10:27:35.003 DEBUG:teuthology.orchestra.run.trial153:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-23T10:27:35.029 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-23T10:27:35.029 INFO:tasks.ceph:role: osd.2 2026-02-23T10:27:35.029 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial153.front.sepia.ceph.com 2026-02-23T10:27:35.029 DEBUG:teuthology.orchestra.run.trial153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-23T10:27:35.093 INFO:teuthology.orchestra.run.trial153.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-23T10:27:35.094 INFO:teuthology.orchestra.run.trial153.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-23T10:27:35.094 INFO:teuthology.orchestra.run.trial153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-23T10:27:35.094 INFO:teuthology.orchestra.run.trial153.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-23T10:27:35.094 INFO:teuthology.orchestra.run.trial153.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-23T10:27:35.094 INFO:teuthology.orchestra.run.trial153.stdout: = sunit=0 swidth=0 blks 2026-02-23T10:27:35.094 INFO:teuthology.orchestra.run.trial153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-23T10:27:35.094 INFO:teuthology.orchestra.run.trial153.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-23T10:27:35.094 INFO:teuthology.orchestra.run.trial153.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-23T10:27:35.094 INFO:teuthology.orchestra.run.trial153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-23T10:27:35.665 INFO:teuthology.orchestra.run.trial153.stdout:Discarding blocks...Done. 2026-02-23T10:27:35.667 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial153.front.sepia.ceph.com -o noatime 2026-02-23T10:27:35.667 DEBUG:teuthology.orchestra.run.trial153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2026-02-23T10:27:35.736 DEBUG:teuthology.orchestra.run.trial153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-23T10:27:35.803 DEBUG:teuthology.orchestra.run.trial153:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-23T10:27:35.866 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-23T10:27:35.867 INFO:tasks.ceph:role: osd.3 2026-02-23T10:27:35.867 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial153.front.sepia.ceph.com 2026-02-23T10:27:35.867 DEBUG:teuthology.orchestra.run.trial153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-23T10:27:35.931 INFO:teuthology.orchestra.run.trial153.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-23T10:27:35.931 INFO:teuthology.orchestra.run.trial153.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-23T10:27:35.932 INFO:teuthology.orchestra.run.trial153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-23T10:27:35.932 INFO:teuthology.orchestra.run.trial153.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-23T10:27:35.932 INFO:teuthology.orchestra.run.trial153.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-23T10:27:35.932 INFO:teuthology.orchestra.run.trial153.stdout: = sunit=0 swidth=0 blks 2026-02-23T10:27:35.932 INFO:teuthology.orchestra.run.trial153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-23T10:27:35.932 INFO:teuthology.orchestra.run.trial153.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-23T10:27:35.932 INFO:teuthology.orchestra.run.trial153.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-23T10:27:35.932 INFO:teuthology.orchestra.run.trial153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-23T10:27:36.490 INFO:teuthology.orchestra.run.trial153.stdout:Discarding blocks...Done. 2026-02-23T10:27:36.492 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial153.front.sepia.ceph.com -o noatime 2026-02-23T10:27:36.493 DEBUG:teuthology.orchestra.run.trial153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2026-02-23T10:27:36.565 DEBUG:teuthology.orchestra.run.trial153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-23T10:27:36.632 DEBUG:teuthology.orchestra.run.trial153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-23T10:27:36.783 INFO:teuthology.orchestra.run.trial153.stderr:INFO  osd - OSD::main: parsing early config 2026-02-23T10:27:36.836 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:36,836 seastar - Reactor backend: linux-aio 2026-02-23T10:27:36.847 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:36,846 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:36.847 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:36,846 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:36.849 INFO:teuthology.orchestra.run.trial153.stderr:==38925==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-23T10:27:36.904 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:36,903 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-23T10:27:36.904 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:36,903 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-23T10:27:36.904 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:36,903 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-23T10:27:36.905 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:36,903 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-23T10:27:36.905 INFO:teuthology.orchestra.run.trial153.stderr:DEBUG 2026-02-23 10:27:36,903 [shard 0:main] seastar - reactor::drain 2026-02-23T10:27:38.937 INFO:teuthology.orchestra.run.trial153.stderr:----------------------------------------------------- 2026-02-23T10:27:38.938 INFO:teuthology.orchestra.run.trial153.stderr:Suppressions used: 2026-02-23T10:27:38.938 INFO:teuthology.orchestra.run.trial153.stderr: count bytes template 2026-02-23T10:27:38.938 INFO:teuthology.orchestra.run.trial153.stderr: 1 8 MallocExtension::Register 2026-02-23T10:27:38.938 INFO:teuthology.orchestra.run.trial153.stderr:----------------------------------------------------- 2026-02-23T10:27:38.938 INFO:teuthology.orchestra.run.trial153.stderr: 2026-02-23T10:27:38.957 INFO:teuthology.orchestra.run.trial153.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-23T10:27:38.957 INFO:teuthology.orchestra.run.trial153.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-23T10:27:39.004 INFO:teuthology.orchestra.run.trial153.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-23T10:27:39.004 INFO:teuthology.orchestra.run.trial153.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-23T10:27:39.011 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:39,010 seastar - Reactor backend: linux-aio 2026-02-23T10:27:39.024 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:39,023 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:39.024 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:39,023 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:39.029 INFO:teuthology.orchestra.run.trial153.stderr:==38923==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-23T10:27:39.029 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:39,028 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-23T10:27:39.086 INFO:teuthology.orchestra.run.trial153.stderr:DEBUG 2026-02-23 10:27:39,086 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-23T10:27:39.164 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f 0x9b7ebc0 0x9b83ce5 0x9b843c5 0x9c9ff46 0x9ca45f1 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:39.165 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f 0x9c9bd02 0x9c9c048 0x9ca693e 0x77f3d8d 0x7be8169 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:39.165 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f 0x9b81c02 0x9b82197 0x9c9fa83 0x9ca45f1 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:39.310 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x1125e3c 0xf2e3c3 0x9b7ddd8 0x9b7e0d5 0x9b7e348 0x9b7f348 0x9b84049 0x9b843c5 0x9c9ff46 0x9ca45f1 0x9c052e1 0x9c042e1 0x7be9638 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:39.310 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f 0xe06a27 0x1125de5 0xf2e3c3 0x9b7ddd8 0x9b7e0d5 0x9b7e348 0x9b7e5fe 0x9b7ebc0 0x9b83ce5 0x9b843c5 0x9c9ff46 0x9ca45f1 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:39.311 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb052c /lib64/libasan.so.8+0xb1437 0xb69729 0x10912a0 0xf2e3ce 0x9b7dc0b 0x9b7e0d5 0x9b7e348 0x9b7e5fe 0x9b7f0ad 0x9b7f5ab 0x9b7fbdb 0x9b7fd45 0x9c8fd63 0x9ca2b6a 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:41.459 INFO:teuthology.orchestra.run.trial153.stderr:----------------------------------------------------- 2026-02-23T10:27:41.460 INFO:teuthology.orchestra.run.trial153.stderr:Suppressions used: 2026-02-23T10:27:41.460 INFO:teuthology.orchestra.run.trial153.stderr: count bytes template 2026-02-23T10:27:41.460 INFO:teuthology.orchestra.run.trial153.stderr: 1 8 MallocExtension::Register 2026-02-23T10:27:41.460 INFO:teuthology.orchestra.run.trial153.stderr:----------------------------------------------------- 2026-02-23T10:27:41.460 INFO:teuthology.orchestra.run.trial153.stderr: 2026-02-23T10:27:41.492 DEBUG:teuthology.orchestra.run.trial153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-23T10:27:41.517 DEBUG:teuthology.orchestra.run.trial153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-23T10:27:41.666 INFO:teuthology.orchestra.run.trial153.stderr:INFO  osd - OSD::main: parsing early config 2026-02-23T10:27:41.727 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:41,726 seastar - Reactor backend: linux-aio 2026-02-23T10:27:41.745 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:41,744 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:41.745 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:41,744 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:41.747 INFO:teuthology.orchestra.run.trial153.stderr:==38991==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-23T10:27:41.802 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:41,801 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-23T10:27:41.802 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:41,801 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-23T10:27:41.802 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:41,801 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-23T10:27:41.802 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:41,801 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-23T10:27:41.803 INFO:teuthology.orchestra.run.trial153.stderr:DEBUG 2026-02-23 10:27:41,801 [shard 0:main] seastar - reactor::drain 2026-02-23T10:27:43.867 INFO:teuthology.orchestra.run.trial153.stderr:----------------------------------------------------- 2026-02-23T10:27:43.867 INFO:teuthology.orchestra.run.trial153.stderr:Suppressions used: 2026-02-23T10:27:43.867 INFO:teuthology.orchestra.run.trial153.stderr: count bytes template 2026-02-23T10:27:43.868 INFO:teuthology.orchestra.run.trial153.stderr: 1 8 MallocExtension::Register 2026-02-23T10:27:43.868 INFO:teuthology.orchestra.run.trial153.stderr:----------------------------------------------------- 2026-02-23T10:27:43.868 INFO:teuthology.orchestra.run.trial153.stderr: 2026-02-23T10:27:43.887 INFO:teuthology.orchestra.run.trial153.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-23T10:27:43.888 INFO:teuthology.orchestra.run.trial153.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-23T10:27:43.934 INFO:teuthology.orchestra.run.trial153.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-23T10:27:43.934 INFO:teuthology.orchestra.run.trial153.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-23T10:27:43.941 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:43,940 seastar - Reactor backend: linux-aio 2026-02-23T10:27:43.955 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:43,954 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:43.955 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:43,954 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-23T10:27:43.959 INFO:teuthology.orchestra.run.trial153.stderr:==38989==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-23T10:27:43.959 INFO:teuthology.orchestra.run.trial153.stderr:INFO 2026-02-23 10:27:43,959 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-23T10:27:44.017 INFO:teuthology.orchestra.run.trial153.stderr:DEBUG 2026-02-23 10:27:44,016 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-23T10:27:44.095 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f 0xe1d3a8 0xe1d906 0x7be8248 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:44.096 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c32 /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9c864ab 0x9c865bb 0x7be81a6 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:44.096 INFO:teuthology.orchestra.run.trial153.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffb69cb940 0xffffffffb69d3aeb 0xffffffffb69d3f4b 0xffffffffb66910d2 0xffffffffb734d451 0xffffffffb7400ba2 2026-02-23T10:27:44.096 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f 0x9b7dd87 0x9b7e0d5 0x9b7e348 0x9b8006f 0x9c95c77 0x9c95ee6 0x9ca0178 0x9ca45f1 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f9f19a 0x6fa2f9b 0x6fa8b45 0x7482004 0x72a89fa 0x72a8afc 0x72a8bf4 0x735cb16 0x735cbcf 0x735cd82 0x735ceba 0x735cf35 0x735cfba 0x735d115 0x7425e13 0x748208d 0x728f241 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:44.244 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xee75f 0x9a48dbb 0x9b02622 0x9b02738 0x9b02814 0x9b02a90 0x9b03037 0x9b0343b 0x9b036f8 0x9b041b6 0x9c864c2 0x9c865bb 0x7be81a6 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:44.244 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x1125e3c 0xf2e3c3 0x9b7ddd8 0x9b7e0d5 0x9b7e348 0x9b7e5fe 0x9b7f0ad 0x9b7f5ab 0x9b7fbdb 0x9b7fd45 0x9c8fd63 0x9ca2b6a 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:44.244 INFO:teuthology.orchestra.run.trial153.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9f10dfc 0x9d045c3 0x9d40a13 0x9d43fc6 0x9d44234 0x9d4437c 0x9d4483b /lib64/libc.so.6+0x3fc2f 0x9b03909 0x9ca27bf 0x9c052e1 0x9c042e1 0x7be8837 0x7bf23a6 0x7a51d33 0x7a8d6c5 0x711ff75 0x6f3ec5e 0x6f40fc6 0x747ed38 0x72a6837 0x72a6939 0x72a6a31 0x735bce2 0x735bd9b 0x735bf4e 0x735c086 0x735c101 0x735c186 0x735c2e1 0x7425af5 0x747edc1 0x728e178 0x72a6bae 0x73b5274 0x7432e6c 0x7432ee7 0x7432f62 0x74eccd2 0x75e6286 0x9e08ac1 0x9d0cb62 0x9dc6e39 0x9e285fb 0x9f988cc 0x9f98f48 0x26ed527 0xa5ce02e /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-23T10:27:46.357 INFO:teuthology.orchestra.run.trial153.stderr:----------------------------------------------------- 2026-02-23T10:27:46.357 INFO:teuthology.orchestra.run.trial153.stderr:Suppressions used: 2026-02-23T10:27:46.358 INFO:teuthology.orchestra.run.trial153.stderr: count bytes template 2026-02-23T10:27:46.358 INFO:teuthology.orchestra.run.trial153.stderr: 1 8 MallocExtension::Register 2026-02-23T10:27:46.358 INFO:teuthology.orchestra.run.trial153.stderr:----------------------------------------------------- 2026-02-23T10:27:46.358 INFO:teuthology.orchestra.run.trial153.stderr: 2026-02-23T10:27:46.389 DEBUG:teuthology.orchestra.run.trial153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-23T10:27:46.415 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-23T10:27:46.415 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:27:46.415 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-23T10:27:46.441 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:27:46.441 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-23T10:27:46.506 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:27:46.506 DEBUG:teuthology.orchestra.run.trial153:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-23T10:27:46.531 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:27:46.531 DEBUG:teuthology.orchestra.run.trial153:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-23T10:27:46.594 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:27:46.594 DEBUG:teuthology.orchestra.run.trial153:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-23T10:27:46.658 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:27:46.658 DEBUG:teuthology.orchestra.run.trial038:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-23T10:27:46.677 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:27:46.677 DEBUG:teuthology.orchestra.run.trial153:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2026-02-23T10:27:46.714 INFO:tasks.ceph:Adding keys to all mons... 2026-02-23T10:27:46.714 DEBUG:teuthology.orchestra.run.trial038:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-23T10:27:46.739 INFO:teuthology.orchestra.run.trial038.stdout:[osd.0] 2026-02-23T10:27:46.739 INFO:teuthology.orchestra.run.trial038.stdout: key = AQCPK5xpiM7LKxAAi1eKIwsfC5weINigB3LBRA== 2026-02-23T10:27:46.739 INFO:teuthology.orchestra.run.trial038.stdout:[osd.1] 2026-02-23T10:27:46.739 INFO:teuthology.orchestra.run.trial038.stdout: key = AQCUK5xp9EKAIhAAnw5rQO+IgO9x+sbaZCp3IQ== 2026-02-23T10:27:46.740 INFO:teuthology.orchestra.run.trial038.stdout:[mgr.x] 2026-02-23T10:27:46.740 INFO:teuthology.orchestra.run.trial038.stdout: key = AQCLK5xpH0kCHBAAtiV2/zpT/SYba2/0+lMvsg== 2026-02-23T10:27:46.740 INFO:teuthology.orchestra.run.trial038.stdout:[osd.2] 2026-02-23T10:27:46.740 INFO:teuthology.orchestra.run.trial038.stdout: key = AQCbK5xpNNtgBRAA7FRw02ISLhg1R9j0ph+Eew== 2026-02-23T10:27:46.740 INFO:teuthology.orchestra.run.trial038.stdout:[osd.3] 2026-02-23T10:27:46.740 INFO:teuthology.orchestra.run.trial038.stdout: key = AQCgK5xpogY0ARAAWT7aJEpQ6gUXU6wTFA0uJg== 2026-02-23T10:27:46.740 INFO:teuthology.orchestra.run.trial038.stdout:[client.0] 2026-02-23T10:27:46.740 INFO:teuthology.orchestra.run.trial038.stdout: key = AQCLK5xplCzDHhAATzJxckSxk9/pZIfuymRR4w== 2026-02-23T10:27:46.740 INFO:teuthology.orchestra.run.trial038.stdout:[client.1] 2026-02-23T10:27:46.740 INFO:teuthology.orchestra.run.trial038.stdout: key = AQCLK5xpVTrWIRAAVZbNPoT07luGg2gzHRHBBg== 2026-02-23T10:27:46.742 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-23T10:27:46.830 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-23T10:27:46.878 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-23T10:27:46.928 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-23T10:27:46.977 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-23T10:27:47.027 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-23T10:27:47.077 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-23T10:27:47.128 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-23T10:27:47.128 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-23T10:27:47.154 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-23T10:27:47.245 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:47.243+0000 7f1c0102cc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.245 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:47.243+0000 7f1c0102cc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.258 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-23T10:27:47.323 DEBUG:teuthology.orchestra.run.trial038:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-23T10:27:47.380 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-23T10:27:47.380 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-23T10:27:47.381 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-23T10:27:47.422 INFO:tasks.ceph.mon.a:Started 2026-02-23T10:27:47.423 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-23T10:27:47.423 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-23T10:27:47.423 DEBUG:teuthology.orchestra.run.trial153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-23T10:27:47.426 INFO:tasks.ceph.mgr.x:Started 2026-02-23T10:27:47.426 DEBUG:tasks.ceph:set 0 configs 2026-02-23T10:27:47.426 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph config dump 2026-02-23T10:27:47.484 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-23T10:27:47.482+0000 7f67404c2c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.484 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-23T10:27:47.482+0000 7f67404c2c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.484 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-23T10:27:47.482+0000 7f67404c2c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.486 INFO:tasks.ceph.mgr.x.trial153.stderr:2026-02-23T10:27:47.484+0000 7f794dbd8fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.500 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:47.496+0000 7f49518cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.503 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:47.501+0000 7f49518cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.512 INFO:tasks.ceph.mgr.x.trial153.stderr:2026-02-23T10:27:47.510+0000 7f794dbd8fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.512 INFO:tasks.ceph.mgr.x.trial153.stderr:2026-02-23T10:27:47.511+0000 7f794dbd8fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.536 INFO:teuthology.orchestra.run.trial038.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-23T10:27:47.544 INFO:tasks.ceph:Setting crush tunables to default 2026-02-23T10:27:47.544 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd crush tunables default 2026-02-23T10:27:47.620 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:47.615+0000 7f6faf174640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.622 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:47.621+0000 7f6faf174640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.657 INFO:teuthology.orchestra.run.trial038.stderr:adjusted tunables profile to default 2026-02-23T10:27:47.666 INFO:tasks.ceph:check_enable_crimson: True 2026-02-23T10:27:47.666 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-23T10:27:47.666 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-23T10:27:47.741 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:47.736+0000 7f131ba5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.743 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:47.741+0000 7f131ba5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:47.782 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-23T10:27:47.780+0000 7f6737bf1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-23T10:27:47.782 INFO:tasks.ceph.mon.a.trial038.stderr:Please be aware that this feature is experimental, untested, 2026-02-23T10:27:47.782 INFO:tasks.ceph.mon.a.trial038.stderr:unsupported, and may result in data corruption, data loss, 2026-02-23T10:27:47.782 INFO:tasks.ceph.mon.a.trial038.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-23T10:27:47.782 INFO:tasks.ceph.mon.a.trial038.stderr:feature with important data. 2026-02-23T10:27:47.782 INFO:tasks.ceph.mon.a.trial038.stderr: 2026-02-23T10:27:48.348 INFO:tasks.ceph.mgr.x.trial153.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-23T10:27:48.348 INFO:tasks.ceph.mgr.x.trial153.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-23T10:27:48.348 INFO:tasks.ceph.mgr.x.trial153.stderr: from numpy import show_config as show_numpy_config 2026-02-23T10:27:48.669 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-23T10:27:48.669 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:27:48.669 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-23T10:27:48.694 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:27:48.694 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-23T10:27:48.759 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:27:48.759 DEBUG:teuthology.orchestra.run.trial153:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-23T10:27:48.784 DEBUG:teuthology.orchestra.run.trial153:> set -ex 2026-02-23T10:27:48.785 DEBUG:teuthology.orchestra.run.trial153:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-23T10:27:48.848 DEBUG:teuthology.orchestra.run.trial153:> sudo ceph --cluster ceph osd new b3f577b0-4848-4cde-8c10-5bb1cd86de8c 0 2026-02-23T10:27:48.962 INFO:teuthology.orchestra.run.trial153.stderr:2026-02-23T10:27:48.956+0000 7fc02bfe4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:48.964 INFO:teuthology.orchestra.run.trial153.stderr:2026-02-23T10:27:48.963+0000 7fc02bfe4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:49.003 INFO:teuthology.orchestra.run.trial153.stdout:0 2026-02-23T10:27:49.011 DEBUG:teuthology.orchestra.run.trial153:> sudo ceph --cluster ceph osd new b323c0a5-98a3-4b42-930a-6908caaec7af 1 2026-02-23T10:27:49.086 INFO:teuthology.orchestra.run.trial153.stderr:2026-02-23T10:27:49.081+0000 7fe8ae6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:49.088 INFO:teuthology.orchestra.run.trial153.stderr:2026-02-23T10:27:49.086+0000 7fe8ae6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:49.126 INFO:teuthology.orchestra.run.trial153.stdout:1 2026-02-23T10:27:49.134 DEBUG:teuthology.orchestra.run.trial153:> sudo ceph --cluster ceph osd new 6af3461b-f1ca-41dc-be75-a647dd085e9d 2 2026-02-23T10:27:49.212 INFO:teuthology.orchestra.run.trial153.stderr:2026-02-23T10:27:49.208+0000 7feac9b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:49.215 INFO:teuthology.orchestra.run.trial153.stderr:2026-02-23T10:27:49.213+0000 7feac9b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:49.252 INFO:teuthology.orchestra.run.trial153.stdout:2 2026-02-23T10:27:49.260 DEBUG:teuthology.orchestra.run.trial153:> sudo ceph --cluster ceph osd new e6cc4313-3977-41b9-91f9-773b50143d5d 3 2026-02-23T10:27:49.334 INFO:teuthology.orchestra.run.trial153.stderr:2026-02-23T10:27:49.329+0000 7f1ef648b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:49.336 INFO:teuthology.orchestra.run.trial153.stderr:2026-02-23T10:27:49.334+0000 7f1ef648b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:49.373 INFO:teuthology.orchestra.run.trial153.stdout:3 2026-02-23T10:27:49.381 DEBUG:teuthology.orchestra.run.trial038:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-23T10:27:49.410 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-23T10:27:49.410 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-23T10:27:49.452 INFO:tasks.ceph.osd.0:Started 2026-02-23T10:27:49.452 DEBUG:teuthology.orchestra.run.trial038:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-23T10:27:49.481 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-23T10:27:49.481 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-23T10:27:49.523 INFO:tasks.ceph.osd.1:Started 2026-02-23T10:27:49.523 DEBUG:teuthology.orchestra.run.trial153:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-23T10:27:49.551 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-23T10:27:49.551 DEBUG:teuthology.orchestra.run.trial153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-23T10:27:49.593 INFO:tasks.ceph.osd.2:Started 2026-02-23T10:27:49.594 DEBUG:teuthology.orchestra.run.trial153:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2026-02-23T10:27:49.621 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-23T10:27:49.621 DEBUG:teuthology.orchestra.run.trial153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2026-02-23T10:27:49.665 INFO:tasks.ceph.osd.3:Started 2026-02-23T10:27:49.665 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-23T10:27:49.742 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:49.738+0000 7f927515c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:49.744 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:49.743+0000 7f927515c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:49.778 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:27:49.778 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":7,"fsid":"ce093877-a685-4ffe-9065-5da136907d50","created":"2026-02-23T10:27:47.496048+0000","modified":"2026-02-23T10:27:49.372414+0000","last_up_change":"0.000000","last_in_change":"2026-02-23T10:27:49.372414+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"b3f577b0-4848-4cde-8c10-5bb1cd86de8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b323c0a5-98a3-4b42-930a-6908caaec7af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6af3461b-f1ca-41dc-be75-a647dd085e9d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e6cc4313-3977-41b9-91f9-773b50143d5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-23T10:27:49.785 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-23T10:27:49.786 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-23T10:27:50.189 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-23T10:27:50.255 INFO:teuthology.misc.health.trial038.stderr:2026-02-23T10:27:50.250+0000 7f01a4ff5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:50.257 INFO:teuthology.misc.health.trial038.stderr:2026-02-23T10:27:50.255+0000 7f01a4ff5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:50.289 INFO:teuthology.misc.health.trial038.stdout: 2026-02-23T10:27:50.290 INFO:teuthology.misc.health.trial038.stdout:{"epoch":7,"fsid":"ce093877-a685-4ffe-9065-5da136907d50","created":"2026-02-23T10:27:47.496048+0000","modified":"2026-02-23T10:27:49.372414+0000","last_up_change":"0.000000","last_in_change":"2026-02-23T10:27:49.372414+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"b3f577b0-4848-4cde-8c10-5bb1cd86de8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b323c0a5-98a3-4b42-930a-6908caaec7af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6af3461b-f1ca-41dc-be75-a647dd085e9d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e6cc4313-3977-41b9-91f9-773b50143d5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-23T10:27:50.296 DEBUG:teuthology.misc:0 of 4 OSDs are up 2026-02-23T10:27:50.559 INFO:tasks.ceph.mgr.x.trial153.stderr:2026-02-23T10:27:50.558+0000 7f786abe1640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-23T10:27:56.699 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-23T10:27:56.767 INFO:teuthology.misc.health.trial038.stderr:2026-02-23T10:27:56.764+0000 7f8a5ff74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:56.770 INFO:teuthology.misc.health.trial038.stderr:2026-02-23T10:27:56.768+0000 7f8a5ff74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:56.881 INFO:teuthology.misc.health.trial038.stdout: 2026-02-23T10:27:56.881 INFO:teuthology.misc.health.trial038.stdout:{"epoch":11,"fsid":"ce093877-a685-4ffe-9065-5da136907d50","created":"2026-02-23T10:27:47.496048+0000","modified":"2026-02-23T10:27:56.671229+0000","last_up_change":"2026-02-23T10:27:56.671229+0000","last_in_change":"2026-02-23T10:27:49.372414+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"b3f577b0-4848-4cde-8c10-5bb1cd86de8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6802","nonce":228069303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6803","nonce":228069303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6808","nonce":228069303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6809","nonce":228069303}]},"public_addr":"10.20.193.38:6802/228069303","cluster_addr":"10.20.193.38:6803/228069303","heartbeat_back_addr":"10.20.193.38:6808/228069303","heartbeat_front_addr":"10.20.193.38:6809/228069303","state":["exists","up"]},{"osd":1,"uuid":"b323c0a5-98a3-4b42-930a-6908caaec7af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6801","nonce":2414620896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6800","nonce":2414620896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6805","nonce":2414620896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6806","nonce":2414620896}]},"public_addr":"10.20.193.38:6801/2414620896","cluster_addr":"10.20.193.38:6800/2414620896","heartbeat_back_addr":"10.20.193.38:6805/2414620896","heartbeat_front_addr":"10.20.193.38:6806/2414620896","state":["exists","up"]},{"osd":2,"uuid":"6af3461b-f1ca-41dc-be75-a647dd085e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6802","nonce":598354170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6803","nonce":598354170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6810","nonce":598354170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6808","nonce":598354170}]},"public_addr":"10.20.193.153:6802/598354170","cluster_addr":"10.20.193.153:6803/598354170","heartbeat_back_addr":"10.20.193.153:6810/598354170","heartbeat_front_addr":"10.20.193.153:6808/598354170","state":["exists","up"]},{"osd":3,"uuid":"e6cc4313-3977-41b9-91f9-773b50143d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6805","nonce":2589717959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6806","nonce":2589717959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6804","nonce":2589717959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6809","nonce":2589717959}]},"public_addr":"10.20.193.153:6805/2589717959","cluster_addr":"10.20.193.153:6806/2589717959","heartbeat_back_addr":"10.20.193.153:6804/2589717959","heartbeat_front_addr":"10.20.193.153:6809/2589717959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-23T10:27:56.889 DEBUG:teuthology.misc:4 of 4 OSDs are up 2026-02-23T10:27:56.889 INFO:tasks.ceph:Creating RBD pool 2026-02-23T10:27:56.890 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-23T10:27:56.964 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:56.959+0000 7fa040ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:56.967 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:56.965+0000 7fa040ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:57.679 INFO:teuthology.orchestra.run.trial038.stderr:pool 'rbd' created 2026-02-23T10:27:57.694 DEBUG:teuthology.orchestra.run.trial038:> rbd --cluster ceph pool init rbd 2026-02-23T10:27:57.732 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:57.730+0000 7f756b8e3240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:57.732 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:57.730+0000 7f756b8e3240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:27:57.735 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:27:57.734+0000 7f756b8e3240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:00.720 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-23T10:28:00.721 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-23T10:28:00.721 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-23T10:28:00.836 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:00.831+0000 7fcbd4ba4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:00.839 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:00.837+0000 7fcbd4ba4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:00.950 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:00.959 INFO:teuthology.orchestra.run.trial038.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-23T10:28:00.960 INFO:tasks.ceph_manager:config epoch is 1 2026-02-23T10:28:00.960 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-23T10:28:00.960 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-23T10:28:00.960 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-23T10:28:01.036 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.033+0000 7f624e811640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.039 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.037+0000 7f624e811640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.184 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:01.191 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6800","nonce":2597822899},{"type":"v1","addr":"10.20.193.153:6801","nonce":2597822899}]},"active_addr":"10.20.193.153:6801/2597822899","active_change":"2026-02-23T10:27:49.548199+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.153:0","nonce":2960608369}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.153:0","nonce":1677119851}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.153:0","nonce":1632208398}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.153:0","nonce":3244526892}]}]} 2026-02-23T10:28:01.195 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-23T10:28:01.195 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-23T10:28:01.195 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-23T10:28:01.272 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.268+0000 7f6e2765a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.274 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.272+0000 7f6e2765a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.385 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:01.385 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":15,"fsid":"ce093877-a685-4ffe-9065-5da136907d50","created":"2026-02-23T10:27:47.496048+0000","modified":"2026-02-23T10:28:00.686656+0000","last_up_change":"2026-02-23T10:27:56.671229+0000","last_in_change":"2026-02-23T10:27:49.372414+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-23T10:27:57.084604+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2026-02-23T10:27:57.573116+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3f577b0-4848-4cde-8c10-5bb1cd86de8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6802","nonce":228069303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6803","nonce":228069303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6808","nonce":228069303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6809","nonce":228069303}]},"public_addr":"10.20.193.38:6802/228069303","cluster_addr":"10.20.193.38:6803/228069303","heartbeat_back_addr":"10.20.193.38:6808/228069303","heartbeat_front_addr":"10.20.193.38:6809/228069303","state":["exists","up"]},{"osd":1,"uuid":"b323c0a5-98a3-4b42-930a-6908caaec7af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6801","nonce":2414620896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6800","nonce":2414620896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6805","nonce":2414620896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6806","nonce":2414620896}]},"public_addr":"10.20.193.38:6801/2414620896","cluster_addr":"10.20.193.38:6800/2414620896","heartbeat_back_addr":"10.20.193.38:6805/2414620896","heartbeat_front_addr":"10.20.193.38:6806/2414620896","state":["exists","up"]},{"osd":2,"uuid":"6af3461b-f1ca-41dc-be75-a647dd085e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6802","nonce":598354170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6803","nonce":598354170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6810","nonce":598354170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6808","nonce":598354170}]},"public_addr":"10.20.193.153:6802/598354170","cluster_addr":"10.20.193.153:6803/598354170","heartbeat_back_addr":"10.20.193.153:6810/598354170","heartbeat_front_addr":"10.20.193.153:6808/598354170","state":["exists","up"]},{"osd":3,"uuid":"e6cc4313-3977-41b9-91f9-773b50143d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6805","nonce":2589717959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6806","nonce":2589717959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6804","nonce":2589717959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6809","nonce":2589717959}]},"public_addr":"10.20.193.153:6805/2589717959","cluster_addr":"10.20.193.153:6806/2589717959","heartbeat_back_addr":"10.20.193.153:6804/2589717959","heartbeat_front_addr":"10.20.193.153:6809/2589717959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-23T10:28:01.395 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-23T10:28:01.395 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-23T10:28:01.472 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.468+0000 7fceb2ec2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.475 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.473+0000 7fceb2ec2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.587 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:01.588 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":15,"fsid":"ce093877-a685-4ffe-9065-5da136907d50","created":"2026-02-23T10:27:47.496048+0000","modified":"2026-02-23T10:28:00.686656+0000","last_up_change":"2026-02-23T10:27:56.671229+0000","last_in_change":"2026-02-23T10:27:49.372414+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-23T10:27:57.084604+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2026-02-23T10:27:57.573116+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3f577b0-4848-4cde-8c10-5bb1cd86de8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6802","nonce":228069303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6803","nonce":228069303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6808","nonce":228069303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6809","nonce":228069303}]},"public_addr":"10.20.193.38:6802/228069303","cluster_addr":"10.20.193.38:6803/228069303","heartbeat_back_addr":"10.20.193.38:6808/228069303","heartbeat_front_addr":"10.20.193.38:6809/228069303","state":["exists","up"]},{"osd":1,"uuid":"b323c0a5-98a3-4b42-930a-6908caaec7af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6801","nonce":2414620896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6800","nonce":2414620896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6805","nonce":2414620896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6806","nonce":2414620896}]},"public_addr":"10.20.193.38:6801/2414620896","cluster_addr":"10.20.193.38:6800/2414620896","heartbeat_back_addr":"10.20.193.38:6805/2414620896","heartbeat_front_addr":"10.20.193.38:6806/2414620896","state":["exists","up"]},{"osd":2,"uuid":"6af3461b-f1ca-41dc-be75-a647dd085e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6802","nonce":598354170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6803","nonce":598354170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6810","nonce":598354170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6808","nonce":598354170}]},"public_addr":"10.20.193.153:6802/598354170","cluster_addr":"10.20.193.153:6803/598354170","heartbeat_back_addr":"10.20.193.153:6810/598354170","heartbeat_front_addr":"10.20.193.153:6808/598354170","state":["exists","up"]},{"osd":3,"uuid":"e6cc4313-3977-41b9-91f9-773b50143d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6805","nonce":2589717959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6806","nonce":2589717959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6804","nonce":2589717959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6809","nonce":2589717959}]},"public_addr":"10.20.193.153:6805/2589717959","cluster_addr":"10.20.193.153:6806/2589717959","heartbeat_back_addr":"10.20.193.153:6804/2589717959","heartbeat_front_addr":"10.20.193.153:6809/2589717959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-23T10:28:01.598 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-23T10:28:01.598 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-23T10:28:01.599 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-23T10:28:01.599 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-23T10:28:01.684 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.680+0000 7f9bf61a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.685 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.680+0000 7f270a45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.685 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.680+0000 7f2605acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.686 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.681+0000 7efc145fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.687 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.685+0000 7f270a45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.687 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.685+0000 7f9bf61a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.687 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.686+0000 7f2605acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.688 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.686+0000 7efc145fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.706 INFO:teuthology.orchestra.run.trial038.stdout:42949672966 2026-02-23T10:28:01.707 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-23T10:28:01.708 INFO:teuthology.orchestra.run.trial038.stdout:47244640261 2026-02-23T10:28:01.708 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-23T10:28:01.709 INFO:teuthology.orchestra.run.trial038.stdout:47244640261 2026-02-23T10:28:01.710 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-23T10:28:01.711 INFO:teuthology.orchestra.run.trial038.stdout:47244640261 2026-02-23T10:28:01.711 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-23T10:28:01.831 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.826+0000 7fdb638cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.831 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.826+0000 7fa29655c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.832 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.826+0000 7fd4aac5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.834 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.832+0000 7fa29655c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.834 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.832+0000 7fd4aac5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.834 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.832+0000 7fdb638cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.835 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.829+0000 7f16a5a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.837 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:01.835+0000 7f16a5a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:01.949 INFO:teuthology.orchestra.run.trial038.stdout:42949672966 2026-02-23T10:28:01.956 INFO:teuthology.orchestra.run.trial038.stdout:47244640260 2026-02-23T10:28:01.959 INFO:teuthology.orchestra.run.trial038.stdout:47244640260 2026-02-23T10:28:01.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.1 2026-02-23T10:28:01.961 DEBUG:teuthology.parallel:result is None 2026-02-23T10:28:01.965 INFO:teuthology.orchestra.run.trial038.stdout:47244640260 2026-02-23T10:28:01.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640260 for osd.2 2026-02-23T10:28:01.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640260 for osd.3 2026-02-23T10:28:01.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640260 for osd.0 2026-02-23T10:28:02.968 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-23T10:28:02.970 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-23T10:28:02.977 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-23T10:28:03.051 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:03.046+0000 7f39b31dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:03.052 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:03.047+0000 7fafc92cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:03.054 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:03.052+0000 7f39b31dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:03.054 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:03.052+0000 7fafc92cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:03.091 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:03.086+0000 7f7f125e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:03.093 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:03.091+0000 7f7f125e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:03.172 INFO:teuthology.orchestra.run.trial038.stdout:47244640260 2026-02-23T10:28:03.172 INFO:teuthology.orchestra.run.trial038.stdout:47244640260 2026-02-23T10:28:03.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640260 for osd.2 2026-02-23T10:28:03.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640260 for osd.3 2026-02-23T10:28:03.206 INFO:teuthology.orchestra.run.trial038.stdout:47244640260 2026-02-23T10:28:03.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640260 for osd.0 2026-02-23T10:28:04.184 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-23T10:28:04.185 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-23T10:28:04.217 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-23T10:28:04.267 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.262+0000 7f378e25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:04.267 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.262+0000 7f51e750b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:04.270 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.268+0000 7f378e25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:04.270 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.268+0000 7f51e750b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:04.306 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.302+0000 7f3d1f5e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:04.309 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.307+0000 7f3d1f5e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:04.384 INFO:teuthology.orchestra.run.trial038.stdout:47244640261 2026-02-23T10:28:04.390 INFO:teuthology.orchestra.run.trial038.stdout:47244640261 2026-02-23T10:28:04.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.2 2026-02-23T10:28:04.395 DEBUG:teuthology.parallel:result is None 2026-02-23T10:28:04.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.3 2026-02-23T10:28:04.401 DEBUG:teuthology.parallel:result is None 2026-02-23T10:28:04.423 INFO:teuthology.orchestra.run.trial038.stdout:47244640261 2026-02-23T10:28:04.433 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.0 2026-02-23T10:28:04.434 DEBUG:teuthology.parallel:result is None 2026-02-23T10:28:04.434 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-23T10:28:04.434 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-23T10:28:04.553 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.549+0000 7fa1a410b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:04.555 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.553+0000 7fa1a410b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:04.665 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:04.666 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-23T10:28:04.675 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2026-02-23T10:28:03.556862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83888736,"kb_used_data":83888736,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125826400,"statfs":{"total":214748299264,"available":128846233600,"internally_reserved":0,"allocated":85902065664,"data_stored":85902065664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.858080"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.699775+0000","last_change":"2026-02-23T10:28:00.704032+0000","last_active":"2026-02-23T10:28:00.699775+0000","last_peered":"2026-02-23T10:28:00.699775+0000","last_clean":"2026-02-23T10:28:00.699775+0000","last_became_active":"2026-02-23T10:27:58.719686+0000","last_became_peered":"2026-02-23T10:27:58.719686+0000","last_unstale":"2026-02-23T10:28:00.699775+0000","last_undegraded":"2026-02-23T10:28:00.699775+0000","last_fullsized":"2026-02-23T10:28:00.699775+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701685+0000","last_change":"2026-02-23T10:28:00.703973+0000","last_active":"2026-02-23T10:28:00.701685+0000","last_peered":"2026-02-23T10:28:00.701685+0000","last_clean":"2026-02-23T10:28:00.701685+0000","last_became_active":"2026-02-23T10:27:58.727877+0000","last_became_peered":"2026-02-23T10:27:58.727877+0000","last_unstale":"2026-02-23T10:28:00.701685+0000","last_undegraded":"2026-02-23T10:28:00.701685+0000","last_fullsized":"2026-02-23T10:28:00.701685+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702213+0000","last_change":"2026-02-23T10:28:00.704184+0000","last_active":"2026-02-23T10:28:00.702213+0000","last_peered":"2026-02-23T10:28:00.702213+0000","last_clean":"2026-02-23T10:28:00.702213+0000","last_became_active":"2026-02-23T10:27:58.727483+0000","last_became_peered":"2026-02-23T10:27:58.727483+0000","last_unstale":"2026-02-23T10:28:00.702213+0000","last_undegraded":"2026-02-23T10:28:00.702213+0000","last_fullsized":"2026-02-23T10:28:00.702213+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.698577+0000","last_change":"2026-02-23T10:28:00.700444+0000","last_active":"2026-02-23T10:28:00.698577+0000","last_peered":"2026-02-23T10:28:00.698577+0000","last_clean":"2026-02-23T10:28:00.698577+0000","last_became_active":"2026-02-23T10:27:58.716752+0000","last_became_peered":"2026-02-23T10:27:58.716752+0000","last_unstale":"2026-02-23T10:28:00.698577+0000","last_undegraded":"2026-02-23T10:28:00.698577+0000","last_fullsized":"2026-02-23T10:28:00.698577+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702345+0000","last_change":"2026-02-23T10:27:58.735668+0000","last_active":"2026-02-23T10:28:00.702345+0000","last_peered":"2026-02-23T10:28:00.702345+0000","last_clean":"2026-02-23T10:28:00.702345+0000","last_became_active":"2026-02-23T10:27:58.723717+0000","last_became_peered":"2026-02-23T10:27:58.723717+0000","last_unstale":"2026-02-23T10:28:00.702345+0000","last_undegraded":"2026-02-23T10:28:00.702345+0000","last_fullsized":"2026-02-23T10:28:00.702345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.696567+0000","last_change":"2026-02-23T10:28:00.699010+0000","last_active":"2026-02-23T10:28:00.696567+0000","last_peered":"2026-02-23T10:28:00.696567+0000","last_clean":"2026-02-23T10:28:00.696567+0000","last_became_active":"2026-02-23T10:27:58.717627+0000","last_became_peered":"2026-02-23T10:27:58.717627+0000","last_unstale":"2026-02-23T10:28:00.696567+0000","last_undegraded":"2026-02-23T10:28:00.696567+0000","last_fullsized":"2026-02-23T10:28:00.696567+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702984+0000","last_change":"2026-02-23T10:28:00.702984+0000","last_active":"2026-02-23T10:28:00.702984+0000","last_peered":"2026-02-23T10:28:00.702984+0000","last_clean":"2026-02-23T10:28:00.702984+0000","last_became_active":"2026-02-23T10:27:58.714489+0000","last_became_peered":"2026-02-23T10:27:58.714489+0000","last_unstale":"2026-02-23T10:28:00.702984+0000","last_undegraded":"2026-02-23T10:28:00.702984+0000","last_fullsized":"2026-02-23T10:28:00.702984+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701943+0000","last_change":"2026-02-23T10:28:00.705095+0000","last_active":"2026-02-23T10:28:00.701943+0000","last_peered":"2026-02-23T10:28:00.701943+0000","last_clean":"2026-02-23T10:28:00.701943+0000","last_became_active":"2026-02-23T10:27:58.721567+0000","last_became_peered":"2026-02-23T10:27:58.721567+0000","last_unstale":"2026-02-23T10:28:00.701943+0000","last_undegraded":"2026-02-23T10:28:00.701943+0000","last_fullsized":"2026-02-23T10:28:00.701943+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.700063+0000","last_change":"2026-02-23T10:28:00.703735+0000","last_active":"2026-02-23T10:28:00.700063+0000","last_peered":"2026-02-23T10:28:00.700063+0000","last_clean":"2026-02-23T10:28:00.700063+0000","last_became_active":"2026-02-23T10:27:58.713473+0000","last_became_peered":"2026-02-23T10:27:58.713473+0000","last_unstale":"2026-02-23T10:28:00.700063+0000","last_undegraded":"2026-02-23T10:28:00.700063+0000","last_fullsized":"2026-02-23T10:28:00.700063+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634306560,"internally_reserved":0,"allocated":64426917888,"data_stored":64426917888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972560,"kb_used_data":20972560,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456224,"statfs":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972528,"kb_used_data":20972528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456256,"statfs":{"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-23T10:28:04.676 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-23T10:28:04.754 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.749+0000 7fee520c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:04.756 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.754+0000 7fee520c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:04.866 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:04.867 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-23T10:28:04.875 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2026-02-23T10:28:03.556862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83888736,"kb_used_data":83888736,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125826400,"statfs":{"total":214748299264,"available":128846233600,"internally_reserved":0,"allocated":85902065664,"data_stored":85902065664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.858080"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.699775+0000","last_change":"2026-02-23T10:28:00.704032+0000","last_active":"2026-02-23T10:28:00.699775+0000","last_peered":"2026-02-23T10:28:00.699775+0000","last_clean":"2026-02-23T10:28:00.699775+0000","last_became_active":"2026-02-23T10:27:58.719686+0000","last_became_peered":"2026-02-23T10:27:58.719686+0000","last_unstale":"2026-02-23T10:28:00.699775+0000","last_undegraded":"2026-02-23T10:28:00.699775+0000","last_fullsized":"2026-02-23T10:28:00.699775+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701685+0000","last_change":"2026-02-23T10:28:00.703973+0000","last_active":"2026-02-23T10:28:00.701685+0000","last_peered":"2026-02-23T10:28:00.701685+0000","last_clean":"2026-02-23T10:28:00.701685+0000","last_became_active":"2026-02-23T10:27:58.727877+0000","last_became_peered":"2026-02-23T10:27:58.727877+0000","last_unstale":"2026-02-23T10:28:00.701685+0000","last_undegraded":"2026-02-23T10:28:00.701685+0000","last_fullsized":"2026-02-23T10:28:00.701685+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702213+0000","last_change":"2026-02-23T10:28:00.704184+0000","last_active":"2026-02-23T10:28:00.702213+0000","last_peered":"2026-02-23T10:28:00.702213+0000","last_clean":"2026-02-23T10:28:00.702213+0000","last_became_active":"2026-02-23T10:27:58.727483+0000","last_became_peered":"2026-02-23T10:27:58.727483+0000","last_unstale":"2026-02-23T10:28:00.702213+0000","last_undegraded":"2026-02-23T10:28:00.702213+0000","last_fullsized":"2026-02-23T10:28:00.702213+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.698577+0000","last_change":"2026-02-23T10:28:00.700444+0000","last_active":"2026-02-23T10:28:00.698577+0000","last_peered":"2026-02-23T10:28:00.698577+0000","last_clean":"2026-02-23T10:28:00.698577+0000","last_became_active":"2026-02-23T10:27:58.716752+0000","last_became_peered":"2026-02-23T10:27:58.716752+0000","last_unstale":"2026-02-23T10:28:00.698577+0000","last_undegraded":"2026-02-23T10:28:00.698577+0000","last_fullsized":"2026-02-23T10:28:00.698577+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702345+0000","last_change":"2026-02-23T10:27:58.735668+0000","last_active":"2026-02-23T10:28:00.702345+0000","last_peered":"2026-02-23T10:28:00.702345+0000","last_clean":"2026-02-23T10:28:00.702345+0000","last_became_active":"2026-02-23T10:27:58.723717+0000","last_became_peered":"2026-02-23T10:27:58.723717+0000","last_unstale":"2026-02-23T10:28:00.702345+0000","last_undegraded":"2026-02-23T10:28:00.702345+0000","last_fullsized":"2026-02-23T10:28:00.702345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.696567+0000","last_change":"2026-02-23T10:28:00.699010+0000","last_active":"2026-02-23T10:28:00.696567+0000","last_peered":"2026-02-23T10:28:00.696567+0000","last_clean":"2026-02-23T10:28:00.696567+0000","last_became_active":"2026-02-23T10:27:58.717627+0000","last_became_peered":"2026-02-23T10:27:58.717627+0000","last_unstale":"2026-02-23T10:28:00.696567+0000","last_undegraded":"2026-02-23T10:28:00.696567+0000","last_fullsized":"2026-02-23T10:28:00.696567+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702984+0000","last_change":"2026-02-23T10:28:00.702984+0000","last_active":"2026-02-23T10:28:00.702984+0000","last_peered":"2026-02-23T10:28:00.702984+0000","last_clean":"2026-02-23T10:28:00.702984+0000","last_became_active":"2026-02-23T10:27:58.714489+0000","last_became_peered":"2026-02-23T10:27:58.714489+0000","last_unstale":"2026-02-23T10:28:00.702984+0000","last_undegraded":"2026-02-23T10:28:00.702984+0000","last_fullsized":"2026-02-23T10:28:00.702984+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701943+0000","last_change":"2026-02-23T10:28:00.705095+0000","last_active":"2026-02-23T10:28:00.701943+0000","last_peered":"2026-02-23T10:28:00.701943+0000","last_clean":"2026-02-23T10:28:00.701943+0000","last_became_active":"2026-02-23T10:27:58.721567+0000","last_became_peered":"2026-02-23T10:27:58.721567+0000","last_unstale":"2026-02-23T10:28:00.701943+0000","last_undegraded":"2026-02-23T10:28:00.701943+0000","last_fullsized":"2026-02-23T10:28:00.701943+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.700063+0000","last_change":"2026-02-23T10:28:00.703735+0000","last_active":"2026-02-23T10:28:00.700063+0000","last_peered":"2026-02-23T10:28:00.700063+0000","last_clean":"2026-02-23T10:28:00.700063+0000","last_became_active":"2026-02-23T10:27:58.713473+0000","last_became_peered":"2026-02-23T10:27:58.713473+0000","last_unstale":"2026-02-23T10:28:00.700063+0000","last_undegraded":"2026-02-23T10:28:00.700063+0000","last_fullsized":"2026-02-23T10:28:00.700063+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634306560,"internally_reserved":0,"allocated":64426917888,"data_stored":64426917888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972560,"kb_used_data":20972560,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456224,"statfs":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972528,"kb_used_data":20972528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456256,"statfs":{"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-23T10:28:04.877 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-23T10:28:04.877 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-23T10:28:04.877 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-23T10:28:04.877 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-23T10:28:04.955 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.951+0000 7fb52a85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:04.957 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:04.955+0000 7fb52a85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:05.096 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:05.096 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-23T10:28:05.106 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-23T10:28:05.106 INFO:teuthology.run_tasks:Running task workunit... 2026-02-23T10:28:05.140 INFO:tasks.workunit:Pulling workunits from ref e371433fcd1390a0b82cfb18effc422c1f5d8c1d 2026-02-23T10:28:05.141 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-23T10:28:05.141 DEBUG:teuthology.orchestra.run.trial038:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-23T10:28:05.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T10:28:05.159 INFO:teuthology.orchestra.run.trial038.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-23T10:28:05.159 DEBUG:teuthology.orchestra.run.trial038:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-23T10:28:05.215 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-23T10:28:05.216 DEBUG:teuthology.orchestra.run.trial038:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-23T10:28:05.271 INFO:tasks.workunit:timeout=3h 2026-02-23T10:28:05.272 INFO:tasks.workunit:cleanup=True 2026-02-23T10:28:05.272 DEBUG:teuthology.orchestra.run.trial038:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/shraddhaag/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e371433fcd1390a0b82cfb18effc422c1f5d8c1d 2026-02-23T10:28:05.334 INFO:tasks.workunit.client.0.trial038.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-23T10:28:26.178 INFO:tasks.workunit.client.0.trial038.stderr:Note: switching to 'e371433fcd1390a0b82cfb18effc422c1f5d8c1d'. 2026-02-23T10:28:26.178 INFO:tasks.workunit.client.0.trial038.stderr: 2026-02-23T10:28:26.178 INFO:tasks.workunit.client.0.trial038.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-23T10:28:26.178 INFO:tasks.workunit.client.0.trial038.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-23T10:28:26.178 INFO:tasks.workunit.client.0.trial038.stderr:state without impacting any branches by switching back to a branch. 2026-02-23T10:28:26.179 INFO:tasks.workunit.client.0.trial038.stderr: 2026-02-23T10:28:26.179 INFO:tasks.workunit.client.0.trial038.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-23T10:28:26.179 INFO:tasks.workunit.client.0.trial038.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-23T10:28:26.179 INFO:tasks.workunit.client.0.trial038.stderr: 2026-02-23T10:28:26.179 INFO:tasks.workunit.client.0.trial038.stderr: git switch -c 2026-02-23T10:28:26.179 INFO:tasks.workunit.client.0.trial038.stderr: 2026-02-23T10:28:26.179 INFO:tasks.workunit.client.0.trial038.stderr:Or undo this operation with: 2026-02-23T10:28:26.179 INFO:tasks.workunit.client.0.trial038.stderr: 2026-02-23T10:28:26.179 INFO:tasks.workunit.client.0.trial038.stderr: git switch - 2026-02-23T10:28:26.179 INFO:tasks.workunit.client.0.trial038.stderr: 2026-02-23T10:28:26.179 INFO:tasks.workunit.client.0.trial038.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-23T10:28:26.180 INFO:tasks.workunit.client.0.trial038.stderr: 2026-02-23T10:28:26.180 INFO:tasks.workunit.client.0.trial038.stderr:HEAD is now at e371433fcd1 qa/suites/crimson-rados: enable cephadm tests 2026-02-23T10:28:26.181 DEBUG:teuthology.orchestra.run.trial038:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-23T10:28:26.240 INFO:tasks.workunit.client.0.trial038.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-23T10:28:26.242 INFO:tasks.workunit.client.0.trial038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-23T10:28:26.242 INFO:tasks.workunit.client.0.trial038.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-23T10:28:26.340 INFO:tasks.workunit.client.0.trial038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-23T10:28:26.372 INFO:tasks.workunit.client.0.trial038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-23T10:28:26.396 INFO:tasks.workunit.client.0.trial038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-23T10:28:26.397 INFO:tasks.workunit.client.0.trial038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-23T10:28:26.398 INFO:tasks.workunit.client.0.trial038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-23T10:28:26.421 INFO:tasks.workunit.client.0.trial038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-23T10:28:26.425 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-23T10:28:26.425 DEBUG:teuthology.orchestra.run.trial038:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-23T10:28:26.481 INFO:tasks.workunit:Running workunits matching cls/test_cls_cmpomap.sh on client.0... 2026-02-23T10:28:26.483 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2026-02-23T10:28:26.483 DEBUG:teuthology.orchestra.run.trial038:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e371433fcd1390a0b82cfb18effc422c1f5d8c1d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2026-02-23T10:28:26.555 INFO:tasks.workunit.client.0.trial038.stdout:Running main() from gmock_main.cc 2026-02-23T10:28:26.555 INFO:tasks.workunit.client.0.trial038.stdout:[==========] Running 31 tests from 1 test suite. 2026-02-23T10:28:26.555 INFO:tasks.workunit.client.0.trial038.stdout:[----------] Global test environment set-up. 2026-02-23T10:28:27.726 INFO:tasks.workunit.client.0.trial038.stdout:[----------] 31 tests from CmpOmap 2026-02-23T10:28:27.726 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2026-02-23T10:28:27.894 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (168 ms) 2026-02-23T10:28:27.894 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2026-02-23T10:28:27.916 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (21 ms) 2026-02-23T10:28:27.916 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2026-02-23T10:28:27.939 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (22 ms) 2026-02-23T10:28:27.939 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2026-02-23T10:28:27.961 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (22 ms) 2026-02-23T10:28:27.961 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_str 2026-02-23T10:28:28.024 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_str (62 ms) 2026-02-23T10:28:28.024 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2026-02-23T10:28:28.080 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_u64 (56 ms) 2026-02-23T10:28:28.080 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2026-02-23T10:28:28.099 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (18 ms) 2026-02-23T10:28:28.099 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2026-02-23T10:28:28.121 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (21 ms) 2026-02-23T10:28:28.121 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2026-02-23T10:28:28.146 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (25 ms) 2026-02-23T10:28:28.146 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2026-02-23T10:28:28.184 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (38 ms) 2026-02-23T10:28:28.184 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2026-02-23T10:28:28.185 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (0 ms) 2026-02-23T10:28:28.185 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2026-02-23T10:28:28.186 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (0 ms) 2026-02-23T10:28:28.186 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2026-02-23T10:28:28.200 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (14 ms) 2026-02-23T10:28:28.200 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2026-02-23T10:28:28.240 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (40 ms) 2026-02-23T10:28:28.241 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2026-02-23T10:28:28.254 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (13 ms) 2026-02-23T10:28:28.254 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2026-02-23T10:28:28.295 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (40 ms) 2026-02-23T10:28:28.295 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2026-02-23T10:28:28.342 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_set_vals_str (47 ms) 2026-02-23T10:28:28.343 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2026-02-23T10:28:28.390 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (47 ms) 2026-02-23T10:28:28.390 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2026-02-23T10:28:28.412 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (21 ms) 2026-02-23T10:28:28.412 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2026-02-23T10:28:28.429 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (17 ms) 2026-02-23T10:28:28.429 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2026-02-23T10:28:28.430 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (0 ms) 2026-02-23T10:28:28.430 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2026-02-23T10:28:28.431 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (0 ms) 2026-02-23T10:28:28.431 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2026-02-23T10:28:28.445 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (14 ms) 2026-02-23T10:28:28.445 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2026-02-23T10:28:28.459 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (14 ms) 2026-02-23T10:28:28.460 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2026-02-23T10:28:28.513 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (53 ms) 2026-02-23T10:28:28.513 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2026-02-23T10:28:28.572 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (58 ms) 2026-02-23T10:28:28.572 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2026-02-23T10:28:28.590 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (18 ms) 2026-02-23T10:28:28.590 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2026-02-23T10:28:28.605 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (15 ms) 2026-02-23T10:28:28.605 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2026-02-23T10:28:28.606 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (1 ms) 2026-02-23T10:28:28.606 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2026-02-23T10:28:28.607 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (0 ms) 2026-02-23T10:28:28.607 INFO:tasks.workunit.client.0.trial038.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2026-02-23T10:28:28.658 INFO:tasks.workunit.client.0.trial038.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (50 ms) 2026-02-23T10:28:28.658 INFO:tasks.workunit.client.0.trial038.stdout:[----------] 31 tests from CmpOmap (931 ms total) 2026-02-23T10:28:28.658 INFO:tasks.workunit.client.0.trial038.stdout: 2026-02-23T10:28:28.658 INFO:tasks.workunit.client.0.trial038.stdout:[----------] Global test environment tear-down 2026-02-23T10:28:28.728 INFO:tasks.workunit.client.0.trial038.stdout:[==========] 31 tests from 1 test suite ran. (2173 ms total) 2026-02-23T10:28:28.728 INFO:tasks.workunit.client.0.trial038.stdout:[ PASSED ] 31 tests. 2026-02-23T10:28:28.731 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-23T10:28:28.731 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-23T10:28:28.795 INFO:tasks.workunit:Stopping ['cls/test_cls_cmpomap.sh'] on client.0... 2026-02-23T10:28:28.796 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-23T10:28:29.145 DEBUG:teuthology.parallel:result is None 2026-02-23T10:28:29.146 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-23T10:28:29.171 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-23T10:28:29.172 DEBUG:teuthology.orchestra.run.trial038:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-23T10:28:29.228 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-23T10:28:29.228 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-23T10:28:29.246 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-23T10:28:29.246 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-23T10:28:29.343 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:29.339+0000 7f7f520b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:29.345 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:29.344+0000 7f7f520b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:29.458 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:29.459 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-23T10:28:29.467 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2026-02-23T10:28:28.725590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83888736,"kb_used_data":83888736,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125826400,"statfs":{"total":214748299264,"available":128846233600,"internally_reserved":0,"allocated":85902065664,"data_stored":85902065664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.166908"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.699775+0000","last_change":"2026-02-23T10:28:00.704032+0000","last_active":"2026-02-23T10:28:00.699775+0000","last_peered":"2026-02-23T10:28:00.699775+0000","last_clean":"2026-02-23T10:28:00.699775+0000","last_became_active":"2026-02-23T10:27:58.719686+0000","last_became_peered":"2026-02-23T10:27:58.719686+0000","last_unstale":"2026-02-23T10:28:00.699775+0000","last_undegraded":"2026-02-23T10:28:00.699775+0000","last_fullsized":"2026-02-23T10:28:00.699775+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701685+0000","last_change":"2026-02-23T10:28:00.703973+0000","last_active":"2026-02-23T10:28:00.701685+0000","last_peered":"2026-02-23T10:28:00.701685+0000","last_clean":"2026-02-23T10:28:00.701685+0000","last_became_active":"2026-02-23T10:27:58.727877+0000","last_became_peered":"2026-02-23T10:27:58.727877+0000","last_unstale":"2026-02-23T10:28:00.701685+0000","last_undegraded":"2026-02-23T10:28:00.701685+0000","last_fullsized":"2026-02-23T10:28:00.701685+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702213+0000","last_change":"2026-02-23T10:28:00.704184+0000","last_active":"2026-02-23T10:28:00.702213+0000","last_peered":"2026-02-23T10:28:00.702213+0000","last_clean":"2026-02-23T10:28:00.702213+0000","last_became_active":"2026-02-23T10:27:58.727483+0000","last_became_peered":"2026-02-23T10:27:58.727483+0000","last_unstale":"2026-02-23T10:28:00.702213+0000","last_undegraded":"2026-02-23T10:28:00.702213+0000","last_fullsized":"2026-02-23T10:28:00.702213+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.698577+0000","last_change":"2026-02-23T10:28:00.700444+0000","last_active":"2026-02-23T10:28:00.698577+0000","last_peered":"2026-02-23T10:28:00.698577+0000","last_clean":"2026-02-23T10:28:00.698577+0000","last_became_active":"2026-02-23T10:27:58.716752+0000","last_became_peered":"2026-02-23T10:27:58.716752+0000","last_unstale":"2026-02-23T10:28:00.698577+0000","last_undegraded":"2026-02-23T10:28:00.698577+0000","last_fullsized":"2026-02-23T10:28:00.698577+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702345+0000","last_change":"2026-02-23T10:27:58.735668+0000","last_active":"2026-02-23T10:28:00.702345+0000","last_peered":"2026-02-23T10:28:00.702345+0000","last_clean":"2026-02-23T10:28:00.702345+0000","last_became_active":"2026-02-23T10:27:58.723717+0000","last_became_peered":"2026-02-23T10:27:58.723717+0000","last_unstale":"2026-02-23T10:28:00.702345+0000","last_undegraded":"2026-02-23T10:28:00.702345+0000","last_fullsized":"2026-02-23T10:28:00.702345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.696567+0000","last_change":"2026-02-23T10:28:00.699010+0000","last_active":"2026-02-23T10:28:00.696567+0000","last_peered":"2026-02-23T10:28:00.696567+0000","last_clean":"2026-02-23T10:28:00.696567+0000","last_became_active":"2026-02-23T10:27:58.717627+0000","last_became_peered":"2026-02-23T10:27:58.717627+0000","last_unstale":"2026-02-23T10:28:00.696567+0000","last_undegraded":"2026-02-23T10:28:00.696567+0000","last_fullsized":"2026-02-23T10:28:00.696567+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702984+0000","last_change":"2026-02-23T10:28:00.702984+0000","last_active":"2026-02-23T10:28:00.702984+0000","last_peered":"2026-02-23T10:28:00.702984+0000","last_clean":"2026-02-23T10:28:00.702984+0000","last_became_active":"2026-02-23T10:27:58.714489+0000","last_became_peered":"2026-02-23T10:27:58.714489+0000","last_unstale":"2026-02-23T10:28:00.702984+0000","last_undegraded":"2026-02-23T10:28:00.702984+0000","last_fullsized":"2026-02-23T10:28:00.702984+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701943+0000","last_change":"2026-02-23T10:28:00.705095+0000","last_active":"2026-02-23T10:28:00.701943+0000","last_peered":"2026-02-23T10:28:00.701943+0000","last_clean":"2026-02-23T10:28:00.701943+0000","last_became_active":"2026-02-23T10:27:58.721567+0000","last_became_peered":"2026-02-23T10:27:58.721567+0000","last_unstale":"2026-02-23T10:28:00.701943+0000","last_undegraded":"2026-02-23T10:28:00.701943+0000","last_fullsized":"2026-02-23T10:28:00.701943+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.700063+0000","last_change":"2026-02-23T10:28:00.703735+0000","last_active":"2026-02-23T10:28:00.700063+0000","last_peered":"2026-02-23T10:28:00.700063+0000","last_clean":"2026-02-23T10:28:00.700063+0000","last_became_active":"2026-02-23T10:27:58.713473+0000","last_became_peered":"2026-02-23T10:27:58.713473+0000","last_unstale":"2026-02-23T10:28:00.700063+0000","last_undegraded":"2026-02-23T10:28:00.700063+0000","last_fullsized":"2026-02-23T10:28:00.700063+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634306560,"internally_reserved":0,"allocated":64426917888,"data_stored":64426917888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972560,"kb_used_data":20972560,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456224,"statfs":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972528,"kb_used_data":20972528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456256,"statfs":{"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-23T10:28:29.469 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-23T10:28:29.544 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:29.541+0000 7f5941435640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:29.547 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:29.545+0000 7f5941435640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:29.657 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:29.657 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-23T10:28:29.666 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2026-02-23T10:28:29.559639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83888736,"kb_used_data":83888736,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125826400,"statfs":{"total":214748299264,"available":128846233600,"internally_reserved":0,"allocated":85902065664,"data_stored":85902065664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000775"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.699775+0000","last_change":"2026-02-23T10:28:00.704032+0000","last_active":"2026-02-23T10:28:00.699775+0000","last_peered":"2026-02-23T10:28:00.699775+0000","last_clean":"2026-02-23T10:28:00.699775+0000","last_became_active":"2026-02-23T10:27:58.719686+0000","last_became_peered":"2026-02-23T10:27:58.719686+0000","last_unstale":"2026-02-23T10:28:00.699775+0000","last_undegraded":"2026-02-23T10:28:00.699775+0000","last_fullsized":"2026-02-23T10:28:00.699775+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701685+0000","last_change":"2026-02-23T10:28:00.703973+0000","last_active":"2026-02-23T10:28:00.701685+0000","last_peered":"2026-02-23T10:28:00.701685+0000","last_clean":"2026-02-23T10:28:00.701685+0000","last_became_active":"2026-02-23T10:27:58.727877+0000","last_became_peered":"2026-02-23T10:27:58.727877+0000","last_unstale":"2026-02-23T10:28:00.701685+0000","last_undegraded":"2026-02-23T10:28:00.701685+0000","last_fullsized":"2026-02-23T10:28:00.701685+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702213+0000","last_change":"2026-02-23T10:28:00.704184+0000","last_active":"2026-02-23T10:28:00.702213+0000","last_peered":"2026-02-23T10:28:00.702213+0000","last_clean":"2026-02-23T10:28:00.702213+0000","last_became_active":"2026-02-23T10:27:58.727483+0000","last_became_peered":"2026-02-23T10:27:58.727483+0000","last_unstale":"2026-02-23T10:28:00.702213+0000","last_undegraded":"2026-02-23T10:28:00.702213+0000","last_fullsized":"2026-02-23T10:28:00.702213+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.698577+0000","last_change":"2026-02-23T10:28:00.700444+0000","last_active":"2026-02-23T10:28:00.698577+0000","last_peered":"2026-02-23T10:28:00.698577+0000","last_clean":"2026-02-23T10:28:00.698577+0000","last_became_active":"2026-02-23T10:27:58.716752+0000","last_became_peered":"2026-02-23T10:27:58.716752+0000","last_unstale":"2026-02-23T10:28:00.698577+0000","last_undegraded":"2026-02-23T10:28:00.698577+0000","last_fullsized":"2026-02-23T10:28:00.698577+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702345+0000","last_change":"2026-02-23T10:27:58.735668+0000","last_active":"2026-02-23T10:28:00.702345+0000","last_peered":"2026-02-23T10:28:00.702345+0000","last_clean":"2026-02-23T10:28:00.702345+0000","last_became_active":"2026-02-23T10:27:58.723717+0000","last_became_peered":"2026-02-23T10:27:58.723717+0000","last_unstale":"2026-02-23T10:28:00.702345+0000","last_undegraded":"2026-02-23T10:28:00.702345+0000","last_fullsized":"2026-02-23T10:28:00.702345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.696567+0000","last_change":"2026-02-23T10:28:00.699010+0000","last_active":"2026-02-23T10:28:00.696567+0000","last_peered":"2026-02-23T10:28:00.696567+0000","last_clean":"2026-02-23T10:28:00.696567+0000","last_became_active":"2026-02-23T10:27:58.717627+0000","last_became_peered":"2026-02-23T10:27:58.717627+0000","last_unstale":"2026-02-23T10:28:00.696567+0000","last_undegraded":"2026-02-23T10:28:00.696567+0000","last_fullsized":"2026-02-23T10:28:00.696567+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702984+0000","last_change":"2026-02-23T10:28:00.702984+0000","last_active":"2026-02-23T10:28:00.702984+0000","last_peered":"2026-02-23T10:28:00.702984+0000","last_clean":"2026-02-23T10:28:00.702984+0000","last_became_active":"2026-02-23T10:27:58.714489+0000","last_became_peered":"2026-02-23T10:27:58.714489+0000","last_unstale":"2026-02-23T10:28:00.702984+0000","last_undegraded":"2026-02-23T10:28:00.702984+0000","last_fullsized":"2026-02-23T10:28:00.702984+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701943+0000","last_change":"2026-02-23T10:28:00.705095+0000","last_active":"2026-02-23T10:28:00.701943+0000","last_peered":"2026-02-23T10:28:00.701943+0000","last_clean":"2026-02-23T10:28:00.701943+0000","last_became_active":"2026-02-23T10:27:58.721567+0000","last_became_peered":"2026-02-23T10:27:58.721567+0000","last_unstale":"2026-02-23T10:28:00.701943+0000","last_undegraded":"2026-02-23T10:28:00.701943+0000","last_fullsized":"2026-02-23T10:28:00.701943+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.700063+0000","last_change":"2026-02-23T10:28:00.703735+0000","last_active":"2026-02-23T10:28:00.700063+0000","last_peered":"2026-02-23T10:28:00.700063+0000","last_clean":"2026-02-23T10:28:00.700063+0000","last_became_active":"2026-02-23T10:27:58.713473+0000","last_became_peered":"2026-02-23T10:27:58.713473+0000","last_unstale":"2026-02-23T10:28:00.700063+0000","last_undegraded":"2026-02-23T10:28:00.700063+0000","last_fullsized":"2026-02-23T10:28:00.700063+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634306560,"internally_reserved":0,"allocated":64426917888,"data_stored":64426917888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972560,"kb_used_data":20972560,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456224,"statfs":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972528,"kb_used_data":20972528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456256,"statfs":{"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-23T10:28:29.667 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-23T10:28:29.667 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-23T10:28:29.741 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:29.737+0000 7fb983ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:29.743 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:29.742+0000 7fb983ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:29.854 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:29.854 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-23T10:28:29.863 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2026-02-23T10:28:29.559639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83888736,"kb_used_data":83888736,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125826400,"statfs":{"total":214748299264,"available":128846233600,"internally_reserved":0,"allocated":85902065664,"data_stored":85902065664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000775"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.699775+0000","last_change":"2026-02-23T10:28:00.704032+0000","last_active":"2026-02-23T10:28:00.699775+0000","last_peered":"2026-02-23T10:28:00.699775+0000","last_clean":"2026-02-23T10:28:00.699775+0000","last_became_active":"2026-02-23T10:27:58.719686+0000","last_became_peered":"2026-02-23T10:27:58.719686+0000","last_unstale":"2026-02-23T10:28:00.699775+0000","last_undegraded":"2026-02-23T10:28:00.699775+0000","last_fullsized":"2026-02-23T10:28:00.699775+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701685+0000","last_change":"2026-02-23T10:28:00.703973+0000","last_active":"2026-02-23T10:28:00.701685+0000","last_peered":"2026-02-23T10:28:00.701685+0000","last_clean":"2026-02-23T10:28:00.701685+0000","last_became_active":"2026-02-23T10:27:58.727877+0000","last_became_peered":"2026-02-23T10:27:58.727877+0000","last_unstale":"2026-02-23T10:28:00.701685+0000","last_undegraded":"2026-02-23T10:28:00.701685+0000","last_fullsized":"2026-02-23T10:28:00.701685+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702213+0000","last_change":"2026-02-23T10:28:00.704184+0000","last_active":"2026-02-23T10:28:00.702213+0000","last_peered":"2026-02-23T10:28:00.702213+0000","last_clean":"2026-02-23T10:28:00.702213+0000","last_became_active":"2026-02-23T10:27:58.727483+0000","last_became_peered":"2026-02-23T10:27:58.727483+0000","last_unstale":"2026-02-23T10:28:00.702213+0000","last_undegraded":"2026-02-23T10:28:00.702213+0000","last_fullsized":"2026-02-23T10:28:00.702213+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.698577+0000","last_change":"2026-02-23T10:28:00.700444+0000","last_active":"2026-02-23T10:28:00.698577+0000","last_peered":"2026-02-23T10:28:00.698577+0000","last_clean":"2026-02-23T10:28:00.698577+0000","last_became_active":"2026-02-23T10:27:58.716752+0000","last_became_peered":"2026-02-23T10:27:58.716752+0000","last_unstale":"2026-02-23T10:28:00.698577+0000","last_undegraded":"2026-02-23T10:28:00.698577+0000","last_fullsized":"2026-02-23T10:28:00.698577+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702345+0000","last_change":"2026-02-23T10:27:58.735668+0000","last_active":"2026-02-23T10:28:00.702345+0000","last_peered":"2026-02-23T10:28:00.702345+0000","last_clean":"2026-02-23T10:28:00.702345+0000","last_became_active":"2026-02-23T10:27:58.723717+0000","last_became_peered":"2026-02-23T10:27:58.723717+0000","last_unstale":"2026-02-23T10:28:00.702345+0000","last_undegraded":"2026-02-23T10:28:00.702345+0000","last_fullsized":"2026-02-23T10:28:00.702345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.696567+0000","last_change":"2026-02-23T10:28:00.699010+0000","last_active":"2026-02-23T10:28:00.696567+0000","last_peered":"2026-02-23T10:28:00.696567+0000","last_clean":"2026-02-23T10:28:00.696567+0000","last_became_active":"2026-02-23T10:27:58.717627+0000","last_became_peered":"2026-02-23T10:27:58.717627+0000","last_unstale":"2026-02-23T10:28:00.696567+0000","last_undegraded":"2026-02-23T10:28:00.696567+0000","last_fullsized":"2026-02-23T10:28:00.696567+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702984+0000","last_change":"2026-02-23T10:28:00.702984+0000","last_active":"2026-02-23T10:28:00.702984+0000","last_peered":"2026-02-23T10:28:00.702984+0000","last_clean":"2026-02-23T10:28:00.702984+0000","last_became_active":"2026-02-23T10:27:58.714489+0000","last_became_peered":"2026-02-23T10:27:58.714489+0000","last_unstale":"2026-02-23T10:28:00.702984+0000","last_undegraded":"2026-02-23T10:28:00.702984+0000","last_fullsized":"2026-02-23T10:28:00.702984+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701943+0000","last_change":"2026-02-23T10:28:00.705095+0000","last_active":"2026-02-23T10:28:00.701943+0000","last_peered":"2026-02-23T10:28:00.701943+0000","last_clean":"2026-02-23T10:28:00.701943+0000","last_became_active":"2026-02-23T10:27:58.721567+0000","last_became_peered":"2026-02-23T10:27:58.721567+0000","last_unstale":"2026-02-23T10:28:00.701943+0000","last_undegraded":"2026-02-23T10:28:00.701943+0000","last_fullsized":"2026-02-23T10:28:00.701943+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.700063+0000","last_change":"2026-02-23T10:28:00.703735+0000","last_active":"2026-02-23T10:28:00.700063+0000","last_peered":"2026-02-23T10:28:00.700063+0000","last_clean":"2026-02-23T10:28:00.700063+0000","last_became_active":"2026-02-23T10:27:58.713473+0000","last_became_peered":"2026-02-23T10:27:58.713473+0000","last_unstale":"2026-02-23T10:28:00.700063+0000","last_undegraded":"2026-02-23T10:28:00.700063+0000","last_fullsized":"2026-02-23T10:28:00.700063+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634306560,"internally_reserved":0,"allocated":64426917888,"data_stored":64426917888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972560,"kb_used_data":20972560,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456224,"statfs":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972528,"kb_used_data":20972528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456256,"statfs":{"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-23T10:28:29.864 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-23T10:28:29.940 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:29.934+0000 7f47d190b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:29.942 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:29.939+0000 7f47d190b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:30.054 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:30.055 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":18,"fsid":"ce093877-a685-4ffe-9065-5da136907d50","created":"2026-02-23T10:27:47.496048+0000","modified":"2026-02-23T10:28:28.722861+0000","last_up_change":"2026-02-23T10:27:56.671229+0000","last_in_change":"2026-02-23T10:27:49.372414+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-23T10:27:57.084604+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2026-02-23T10:27:57.573116+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3f577b0-4848-4cde-8c10-5bb1cd86de8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6802","nonce":228069303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6803","nonce":228069303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6808","nonce":228069303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6809","nonce":228069303}]},"public_addr":"10.20.193.38:6802/228069303","cluster_addr":"10.20.193.38:6803/228069303","heartbeat_back_addr":"10.20.193.38:6808/228069303","heartbeat_front_addr":"10.20.193.38:6809/228069303","state":["exists","up"]},{"osd":1,"uuid":"b323c0a5-98a3-4b42-930a-6908caaec7af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6801","nonce":2414620896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6800","nonce":2414620896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6805","nonce":2414620896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6806","nonce":2414620896}]},"public_addr":"10.20.193.38:6801/2414620896","cluster_addr":"10.20.193.38:6800/2414620896","heartbeat_back_addr":"10.20.193.38:6805/2414620896","heartbeat_front_addr":"10.20.193.38:6806/2414620896","state":["exists","up"]},{"osd":2,"uuid":"6af3461b-f1ca-41dc-be75-a647dd085e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6802","nonce":598354170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6803","nonce":598354170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6810","nonce":598354170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6808","nonce":598354170}]},"public_addr":"10.20.193.153:6802/598354170","cluster_addr":"10.20.193.153:6803/598354170","heartbeat_back_addr":"10.20.193.153:6810/598354170","heartbeat_front_addr":"10.20.193.153:6808/598354170","state":["exists","up"]},{"osd":3,"uuid":"e6cc4313-3977-41b9-91f9-773b50143d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6805","nonce":2589717959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6806","nonce":2589717959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6804","nonce":2589717959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.153:6809","nonce":2589717959}]},"public_addr":"10.20.193.153:6805/2589717959","cluster_addr":"10.20.193.153:6806/2589717959","heartbeat_back_addr":"10.20.193.153:6804/2589717959","heartbeat_front_addr":"10.20.193.153:6809/2589717959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-23T10:28:31.065 INFO:tasks.ceph:Scrubbing osd.0 2026-02-23T10:28:31.065 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-23T10:28:31.147 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.141+0000 7f80efccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.150 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.148+0000 7f80efccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.160 INFO:teuthology.orchestra.run.trial038.stdout:{ 2026-02-23T10:28:31.160 INFO:teuthology.orchestra.run.trial038.stdout: "success": "" 2026-02-23T10:28:31.160 INFO:teuthology.orchestra.run.trial038.stdout:} 2026-02-23T10:28:31.170 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-23T10:28:31.246 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.242+0000 7fc54c3d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.249 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.247+0000 7fc54c3d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.362 INFO:teuthology.orchestra.run.trial038.stderr:instructed osd(s) 0 to deep-scrub 2026-02-23T10:28:31.371 INFO:tasks.ceph:Scrubbing osd.1 2026-02-23T10:28:31.371 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-23T10:28:31.451 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.444+0000 7ff3dad64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.453 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.451+0000 7ff3dad64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.463 INFO:teuthology.orchestra.run.trial038.stdout:{ 2026-02-23T10:28:31.463 INFO:teuthology.orchestra.run.trial038.stdout: "success": "" 2026-02-23T10:28:31.463 INFO:teuthology.orchestra.run.trial038.stdout:} 2026-02-23T10:28:31.471 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-23T10:28:31.549 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.545+0000 7f2de7dfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.551 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.550+0000 7f2de7dfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.663 INFO:teuthology.orchestra.run.trial038.stderr:instructed osd(s) 1 to deep-scrub 2026-02-23T10:28:31.674 INFO:tasks.ceph:Scrubbing osd.2 2026-02-23T10:28:31.675 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-23T10:28:31.753 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.747+0000 7f40ef61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.756 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.754+0000 7f40ef61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.766 INFO:teuthology.orchestra.run.trial038.stdout:{ 2026-02-23T10:28:31.766 INFO:teuthology.orchestra.run.trial038.stdout: "success": "" 2026-02-23T10:28:31.766 INFO:teuthology.orchestra.run.trial038.stdout:} 2026-02-23T10:28:31.773 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-23T10:28:31.850 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.846+0000 7fbdbb993640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.853 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:31.851+0000 7fbdbb993640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:31.964 INFO:teuthology.orchestra.run.trial038.stderr:instructed osd(s) 2 to deep-scrub 2026-02-23T10:28:31.973 INFO:tasks.ceph:Scrubbing osd.3 2026-02-23T10:28:31.973 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-23T10:28:32.055 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:32.049+0000 7f019b45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:32.058 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:32.056+0000 7f019b45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:32.068 INFO:teuthology.orchestra.run.trial038.stdout:{ 2026-02-23T10:28:32.069 INFO:teuthology.orchestra.run.trial038.stdout: "success": "" 2026-02-23T10:28:32.069 INFO:teuthology.orchestra.run.trial038.stdout:} 2026-02-23T10:28:32.076 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-23T10:28:32.154 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:32.149+0000 7f79224cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:32.156 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:32.155+0000 7f79224cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:32.267 INFO:teuthology.orchestra.run.trial038.stderr:instructed osd(s) 3 to deep-scrub 2026-02-23T10:28:32.277 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-23T10:28:32.352 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:32.348+0000 7f033d8cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:32.355 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:32.353+0000 7f033d8cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:32.468 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:32.469 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-23T10:28:32.478 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2026-02-23T10:28:31.559971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83888788,"kb_used_data":83888788,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125826348,"statfs":{"total":214748299264,"available":128846180352,"internally_reserved":0,"allocated":85902118912,"data_stored":85902118912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000908"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.699775+0000","last_change":"2026-02-23T10:28:00.704032+0000","last_active":"2026-02-23T10:28:00.699775+0000","last_peered":"2026-02-23T10:28:00.699775+0000","last_clean":"2026-02-23T10:28:00.699775+0000","last_became_active":"2026-02-23T10:27:58.719686+0000","last_became_peered":"2026-02-23T10:27:58.719686+0000","last_unstale":"2026-02-23T10:28:00.699775+0000","last_undegraded":"2026-02-23T10:28:00.699775+0000","last_fullsized":"2026-02-23T10:28:00.699775+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701685+0000","last_change":"2026-02-23T10:28:00.703973+0000","last_active":"2026-02-23T10:28:00.701685+0000","last_peered":"2026-02-23T10:28:00.701685+0000","last_clean":"2026-02-23T10:28:00.701685+0000","last_became_active":"2026-02-23T10:27:58.727877+0000","last_became_peered":"2026-02-23T10:27:58.727877+0000","last_unstale":"2026-02-23T10:28:00.701685+0000","last_undegraded":"2026-02-23T10:28:00.701685+0000","last_fullsized":"2026-02-23T10:28:00.701685+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702213+0000","last_change":"2026-02-23T10:28:00.704184+0000","last_active":"2026-02-23T10:28:00.702213+0000","last_peered":"2026-02-23T10:28:00.702213+0000","last_clean":"2026-02-23T10:28:00.702213+0000","last_became_active":"2026-02-23T10:27:58.727483+0000","last_became_peered":"2026-02-23T10:27:58.727483+0000","last_unstale":"2026-02-23T10:28:00.702213+0000","last_undegraded":"2026-02-23T10:28:00.702213+0000","last_fullsized":"2026-02-23T10:28:00.702213+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-23T10:28:28.743929+0000","last_change":"2026-02-23T10:28:00.700444+0000","last_active":"2026-02-23T10:28:28.743929+0000","last_peered":"2026-02-23T10:28:28.743929+0000","last_clean":"2026-02-23T10:28:28.743929+0000","last_became_active":"2026-02-23T10:27:58.716752+0000","last_became_peered":"2026-02-23T10:27:58.716752+0000","last_unstale":"2026-02-23T10:28:28.743929+0000","last_undegraded":"2026-02-23T10:28:28.743929+0000","last_fullsized":"2026-02-23T10:28:28.743929+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702345+0000","last_change":"2026-02-23T10:27:58.735668+0000","last_active":"2026-02-23T10:28:00.702345+0000","last_peered":"2026-02-23T10:28:00.702345+0000","last_clean":"2026-02-23T10:28:00.702345+0000","last_became_active":"2026-02-23T10:27:58.723717+0000","last_became_peered":"2026-02-23T10:27:58.723717+0000","last_unstale":"2026-02-23T10:28:00.702345+0000","last_undegraded":"2026-02-23T10:28:00.702345+0000","last_fullsized":"2026-02-23T10:28:00.702345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":25,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-23T10:28:28.743247+0000","last_change":"2026-02-23T10:28:00.699010+0000","last_active":"2026-02-23T10:28:28.743247+0000","last_peered":"2026-02-23T10:28:28.743247+0000","last_clean":"2026-02-23T10:28:28.743247+0000","last_became_active":"2026-02-23T10:27:58.717627+0000","last_became_peered":"2026-02-23T10:27:58.717627+0000","last_unstale":"2026-02-23T10:28:28.743247+0000","last_undegraded":"2026-02-23T10:28:28.743247+0000","last_fullsized":"2026-02-23T10:28:28.743247+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.702984+0000","last_change":"2026-02-23T10:28:00.702984+0000","last_active":"2026-02-23T10:28:00.702984+0000","last_peered":"2026-02-23T10:28:00.702984+0000","last_clean":"2026-02-23T10:28:00.702984+0000","last_became_active":"2026-02-23T10:27:58.714489+0000","last_became_peered":"2026-02-23T10:27:58.714489+0000","last_unstale":"2026-02-23T10:28:00.702984+0000","last_undegraded":"2026-02-23T10:28:00.702984+0000","last_fullsized":"2026-02-23T10:28:00.702984+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.701943+0000","last_change":"2026-02-23T10:28:00.705095+0000","last_active":"2026-02-23T10:28:00.701943+0000","last_peered":"2026-02-23T10:28:00.701943+0000","last_clean":"2026-02-23T10:28:00.701943+0000","last_became_active":"2026-02-23T10:27:58.721567+0000","last_became_peered":"2026-02-23T10:27:58.721567+0000","last_unstale":"2026-02-23T10:28:00.701943+0000","last_undegraded":"2026-02-23T10:28:00.701943+0000","last_fullsized":"2026-02-23T10:28:00.701943+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-23T10:28:00.700063+0000","last_change":"2026-02-23T10:28:00.703735+0000","last_active":"2026-02-23T10:28:00.700063+0000","last_peered":"2026-02-23T10:28:00.700063+0000","last_clean":"2026-02-23T10:28:00.700063+0000","last_became_active":"2026-02-23T10:27:58.713473+0000","last_became_peered":"2026-02-23T10:27:58.713473+0000","last_unstale":"2026-02-23T10:28:00.700063+0000","last_undegraded":"2026-02-23T10:28:00.700063+0000","last_fullsized":"2026-02-23T10:28:00.700063+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:27:57.675287+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634253312,"internally_reserved":0,"allocated":64426971136,"data_stored":64426971136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972560,"kb_used_data":20972560,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456224,"statfs":{"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972580,"kb_used_data":20972580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456204,"statfs":{"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32211173376,"internally_reserved":0,"allocated":21475901440,"data_stored":21475901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-23T10:28:32.483 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-23T10:27:57.675287+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=27, tm_sec=57, tm_wday=0, tm_yday=54, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=28, tm_sec=30, tm_wday=0, tm_yday=54, tm_isdst=0) 2026-02-23T10:28:32.483 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-23T10:27:57.675287+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=27, tm_sec=57, tm_wday=0, tm_yday=54, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=28, tm_sec=30, tm_wday=0, tm_yday=54, tm_isdst=0) 2026-02-23T10:28:32.483 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-23T10:27:57.675287+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=27, tm_sec=57, tm_wday=0, tm_yday=54, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=28, tm_sec=30, tm_wday=0, tm_yday=54, tm_isdst=0) 2026-02-23T10:28:32.483 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-23T10:27:57.675287+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=27, tm_sec=57, tm_wday=0, tm_yday=54, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=28, tm_sec=30, tm_wday=0, tm_yday=54, tm_isdst=0) 2026-02-23T10:28:32.483 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-23T10:27:57.675287+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=27, tm_sec=57, tm_wday=0, tm_yday=54, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=28, tm_sec=30, tm_wday=0, tm_yday=54, tm_isdst=0) 2026-02-23T10:28:32.484 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-23T10:27:57.675287+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=27, tm_sec=57, tm_wday=0, tm_yday=54, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=28, tm_sec=30, tm_wday=0, tm_yday=54, tm_isdst=0) 2026-02-23T10:28:32.484 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2026-02-23T10:27:57.675287+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=27, tm_sec=57, tm_wday=0, tm_yday=54, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=28, tm_sec=30, tm_wday=0, tm_yday=54, tm_isdst=0) 2026-02-23T10:28:32.484 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-23T10:27:57.675287+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=27, tm_sec=57, tm_wday=0, tm_yday=54, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=28, tm_sec=30, tm_wday=0, tm_yday=54, tm_isdst=0) 2026-02-23T10:28:32.484 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-23T10:27:57.675287+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=27, tm_sec=57, tm_wday=0, tm_yday=54, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=23, tm_hour=10, tm_min=28, tm_sec=30, tm_wday=0, tm_yday=54, tm_isdst=0) 2026-02-23T10:28:32.485 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-23T10:28:52.486 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-23T10:28:52.565 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:52.561+0000 7f511445a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:52.567 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:52.566+0000 7f511445a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:52.681 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-23T10:28:52.682 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-23T10:28:52.691 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2026-02-23T10:28:51.563423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83888944,"kb_used_data":83888944,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125826192,"statfs":{"total":214748299264,"available":128846020608,"internally_reserved":0,"allocated":85902278656,"data_stored":85902278656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002020"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-23T10:28:32.270046+0000","last_change":"2026-02-23T10:28:32.281006+0000","last_active":"2026-02-23T10:28:32.270046+0000","last_peered":"2026-02-23T10:28:32.270046+0000","last_clean":"2026-02-23T10:28:32.270046+0000","last_became_active":"2026-02-23T10:27:58.719686+0000","last_became_peered":"2026-02-23T10:27:58.719686+0000","last_unstale":"2026-02-23T10:28:32.270046+0000","last_undegraded":"2026-02-23T10:28:32.270046+0000","last_fullsized":"2026-02-23T10:28:32.270046+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:28:32.280948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:28:32.280948+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-23T10:28:32.269572+0000","last_change":"2026-02-23T10:28:32.280872+0000","last_active":"2026-02-23T10:28:32.269572+0000","last_peered":"2026-02-23T10:28:32.269572+0000","last_clean":"2026-02-23T10:28:32.269572+0000","last_became_active":"2026-02-23T10:27:58.727877+0000","last_became_peered":"2026-02-23T10:27:58.727877+0000","last_unstale":"2026-02-23T10:28:32.269572+0000","last_undegraded":"2026-02-23T10:28:32.269572+0000","last_fullsized":"2026-02-23T10:28:32.269572+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:28:32.280798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:28:32.280798+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-23T10:28:31.966028+0000","last_change":"2026-02-23T10:28:31.974040+0000","last_active":"2026-02-23T10:28:31.966028+0000","last_peered":"2026-02-23T10:28:31.966028+0000","last_clean":"2026-02-23T10:28:31.966028+0000","last_became_active":"2026-02-23T10:27:58.727483+0000","last_became_peered":"2026-02-23T10:27:58.727483+0000","last_unstale":"2026-02-23T10:28:31.966028+0000","last_undegraded":"2026-02-23T10:28:31.966028+0000","last_fullsized":"2026-02-23T10:28:31.966028+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:28:31.973941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:28:31.973941+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-23T10:28:31.664562+0000","last_change":"2026-02-23T10:28:31.672425+0000","last_active":"2026-02-23T10:28:31.664562+0000","last_peered":"2026-02-23T10:28:31.664562+0000","last_clean":"2026-02-23T10:28:31.664562+0000","last_became_active":"2026-02-23T10:27:58.716752+0000","last_became_peered":"2026-02-23T10:27:58.716752+0000","last_unstale":"2026-02-23T10:28:31.664562+0000","last_undegraded":"2026-02-23T10:28:31.664562+0000","last_fullsized":"2026-02-23T10:28:31.664562+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:28:31.672347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:28:31.672347+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":92,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-23T10:28:32.271325+0000","last_change":"2026-02-23T10:28:32.288280+0000","last_active":"2026-02-23T10:28:32.271325+0000","last_peered":"2026-02-23T10:28:32.271325+0000","last_clean":"2026-02-23T10:28:32.271325+0000","last_became_active":"2026-02-23T10:27:58.723717+0000","last_became_peered":"2026-02-23T10:27:58.723717+0000","last_unstale":"2026-02-23T10:28:32.271325+0000","last_undegraded":"2026-02-23T10:28:32.271325+0000","last_fullsized":"2026-02-23T10:28:32.271325+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"13'52","last_scrub_stamp":"2026-02-23T10:28:32.288193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:28:32.288193+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":27,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-23T10:28:31.664433+0000","last_change":"2026-02-23T10:28:31.675371+0000","last_active":"2026-02-23T10:28:31.664433+0000","last_peered":"2026-02-23T10:28:31.664433+0000","last_clean":"2026-02-23T10:28:31.664433+0000","last_became_active":"2026-02-23T10:27:58.717627+0000","last_became_peered":"2026-02-23T10:27:58.717627+0000","last_unstale":"2026-02-23T10:28:31.664433+0000","last_undegraded":"2026-02-23T10:28:31.664433+0000","last_fullsized":"2026-02-23T10:28:31.664433+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"13'1","last_scrub_stamp":"2026-02-23T10:28:31.675311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:28:31.675311+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-23T10:28:31.964473+0000","last_change":"2026-02-23T10:28:31.977813+0000","last_active":"2026-02-23T10:28:31.964473+0000","last_peered":"2026-02-23T10:28:31.964473+0000","last_clean":"2026-02-23T10:28:31.964473+0000","last_became_active":"2026-02-23T10:27:58.714489+0000","last_became_peered":"2026-02-23T10:27:58.714489+0000","last_unstale":"2026-02-23T10:28:31.964473+0000","last_undegraded":"2026-02-23T10:28:31.964473+0000","last_fullsized":"2026-02-23T10:28:31.964473+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2026-02-23T10:28:31.977736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:28:31.977736+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-23T10:28:32.269000+0000","last_change":"2026-02-23T10:28:32.279831+0000","last_active":"2026-02-23T10:28:32.269000+0000","last_peered":"2026-02-23T10:28:32.269000+0000","last_clean":"2026-02-23T10:28:32.269000+0000","last_became_active":"2026-02-23T10:27:58.721567+0000","last_became_peered":"2026-02-23T10:27:58.721567+0000","last_unstale":"2026-02-23T10:28:32.269000+0000","last_undegraded":"2026-02-23T10:28:32.269000+0000","last_fullsized":"2026-02-23T10:28:32.269000+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:28:32.279723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:28:32.279723+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-23T10:28:32.268096+0000","last_change":"2026-02-23T10:28:32.279854+0000","last_active":"2026-02-23T10:28:32.268096+0000","last_peered":"2026-02-23T10:28:32.268096+0000","last_clean":"2026-02-23T10:28:32.268096+0000","last_became_active":"2026-02-23T10:27:58.713473+0000","last_became_peered":"2026-02-23T10:27:58.713473+0000","last_unstale":"2026-02-23T10:28:32.268096+0000","last_undegraded":"2026-02-23T10:28:32.268096+0000","last_fullsized":"2026-02-23T10:28:32.268096+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-23T10:28:32.279716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-23T10:28:32.279716+0000","last_clean_scrub_stamp":"2026-02-23T10:27:57.675287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211120128,"internally_reserved":0,"allocated":21475954688,"data_stored":21475954688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634146816,"internally_reserved":0,"allocated":64427077632,"data_stored":64427077632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640305,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972612,"kb_used_data":20972612,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456172,"statfs":{"total":53687074816,"available":32211120128,"internally_reserved":0,"allocated":21475954688,"data_stored":21475954688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640305,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971876,"kb_used_data":20971876,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456908,"statfs":{"total":53687074816,"available":32211873792,"internally_reserved":0,"allocated":21475201024,"data_stored":21475201024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972580,"kb_used_data":20972580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456204,"statfs":{"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640305,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971876,"kb_used_data":20971876,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456908,"statfs":{"total":53687074816,"available":32211873792,"internally_reserved":0,"allocated":21475201024,"data_stored":21475201024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53687074816,"available":32211873792,"internally_reserved":0,"allocated":21475201024,"data_stored":21475201024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53687074816,"available":32211120128,"internally_reserved":0,"allocated":21475954688,"data_stored":21475954688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32211120128,"internally_reserved":0,"allocated":21475954688,"data_stored":21475954688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-23T10:28:52.693 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-23T10:28:52.769 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:52.764+0000 7f38e465a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:52.772 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-23T10:28:52.770+0000 7f38e465a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-23T10:28:52.896 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-23T10:28:52.896 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-23T10:28:52.897 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-23T10:28:52.897 INFO:teuthology.orchestra.run:waiting for 300 2026-02-23T10:28:52.956 INFO:tasks.ceph.osd.0:Stopped 2026-02-23T10:28:52.957 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-23T10:28:52.957 INFO:teuthology.orchestra.run:waiting for 300 2026-02-23T10:28:53.004 INFO:tasks.ceph.osd.1:Stopped 2026-02-23T10:28:53.005 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-23T10:28:53.005 INFO:teuthology.orchestra.run:waiting for 300 2026-02-23T10:28:53.052 INFO:tasks.ceph.osd.2:Stopped 2026-02-23T10:28:53.052 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-23T10:28:53.053 INFO:teuthology.orchestra.run:waiting for 300 2026-02-23T10:28:53.107 INFO:tasks.ceph.osd.3:Stopped 2026-02-23T10:28:53.107 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-23T10:28:53.107 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-23T10:28:53.107 INFO:teuthology.orchestra.run:waiting for 300 2026-02-23T10:28:53.139 INFO:tasks.ceph.mgr.x:Stopped 2026-02-23T10:28:53.139 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-23T10:28:53.140 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-23T10:28:53.140 INFO:teuthology.orchestra.run:waiting for 300 2026-02-23T10:28:53.149 INFO:tasks.ceph.mon.a:Stopped 2026-02-23T10:28:53.150 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-23T10:28:53.150 DEBUG:teuthology.orchestra.run.trial038:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2026-02-23T10:28:53.215 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial038.front.sepia.ceph.com 2026-02-23T10:28:53.215 DEBUG:teuthology.orchestra.run.trial038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-23T10:28:53.284 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial038.front.sepia.ceph.com 2026-02-23T10:28:53.284 DEBUG:teuthology.orchestra.run.trial038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-23T10:28:53.351 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial153.front.sepia.ceph.com 2026-02-23T10:28:53.351 DEBUG:teuthology.orchestra.run.trial153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-23T10:28:53.381 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial153.front.sepia.ceph.com 2026-02-23T10:28:53.381 DEBUG:teuthology.orchestra.run.trial153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-23T10:28:53.498 INFO:tasks.ceph:Archiving mon data... 2026-02-23T10:28:53.500 DEBUG:teuthology.misc:Transferring archived files from trial038:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial/66074/data/mon.a.tgz 2026-02-23T10:28:53.500 DEBUG:teuthology.orchestra.run.trial038:> mktemp 2026-02-23T10:28:53.518 INFO:teuthology.orchestra.run.trial038.stdout:/tmp/tmp.Q0mJfdgNre 2026-02-23T10:28:53.519 DEBUG:teuthology.orchestra.run.trial038:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Q0mJfdgNre 2026-02-23T10:28:53.603 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 0666 /tmp/tmp.Q0mJfdgNre 2026-02-23T10:28:53.725 DEBUG:teuthology.orchestra.remote:trial038:/tmp/tmp.Q0mJfdgNre is 170KB 2026-02-23T10:28:53.776 DEBUG:teuthology.orchestra.run.trial038:> rm -fr /tmp/tmp.Q0mJfdgNre 2026-02-23T10:28:53.792 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-23T10:28:53.793 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-23T10:28:53.835 DEBUG:teuthology.orchestra.run.trial153:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-23T10:28:53.915 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-23T10:28:53.932 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-23T10:28:53.933 INFO:tasks.ceph:Archiving crash dumps... 2026-02-23T10:28:53.935 DEBUG:teuthology.misc:Transferring archived files from trial038:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial/66074/remote/trial038/crash 2026-02-23T10:28:53.936 DEBUG:teuthology.orchestra.run.trial038:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-23T10:28:53.980 DEBUG:teuthology.misc:Transferring archived files from trial153:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial/66074/remote/trial153/crash 2026-02-23T10:28:53.981 DEBUG:teuthology.orchestra.run.trial153:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-23T10:28:54.007 INFO:tasks.ceph:Compressing logs... 2026-02-23T10:28:54.008 DEBUG:teuthology.orchestra.run.trial038:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-23T10:28:54.025 DEBUG:teuthology.orchestra.run.trial153:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-23T10:28:54.047 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-23T10:28:54.064 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41674.log 2026-02-23T10:28:54.064 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40667.log 2026-02-23T10:28:54.064 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41674.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-23T10:28:54.064 INFO:teuthology.orchestra.run.trial038.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41674.log.gz 2026-02-23T10:28:54.064 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41233.log 2026-02-23T10:28:54.064 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40667.log.gz 2026-02-23T10:28:54.064 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40351.log 2026-02-23T10:28:54.064 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41576.log 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.41233.log.gz 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40351.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40351.log.gz 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41184.log 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39785.log 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41576.log.gz 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41184.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41184.log.gz 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40716.log 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.39785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40497.log 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39785.log.gz 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40525.log 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40716.log.gz 2026-02-23T10:28:54.065 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39883.log 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40497.log.gz 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39736.log 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40525.log.gz 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.39883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39636.log 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39883.log.gz 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.39736.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-client.admin.41773.log 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39736.log.gz 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.39636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40010.log 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39636.log.gz 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40208.log 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41773.log.gz 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39225.log 2026-02-23T10:28:54.066 INFO:teuthology.orchestra.run.trial038.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40010.log.gz 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39139.log 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40208.log.gz 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.39225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39834.log 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39225.log.gz 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.39139.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.39139.log.gz 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41135.log 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.39834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40209.log 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39834.log.gz 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40012.log 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41135.log.gz 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40206.log 2026-02-23T10:28:54.067 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40209.log.gz 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40011.log 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40012.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40012.log.gz 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40013.log 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40206.log.gz 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39506.log 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40011.log.gz 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40013.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40013.log.gz 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.39506.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39506.log.gz 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41331.log 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41478.log 2026-02-23T10:28:54.068 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41331.log: 87.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41331.log.gzgzip 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38080.log 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41478.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41478.log.gz 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41380.log 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph.tmp-client.admin.38080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39663.log 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38080.log.gz 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39272.log 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41380.log.gz 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.39663.log: -- /var/log/ceph/ceph-client.admin.40990.log 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.39663.log.gz 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.39272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41527.log 2026-02-23T10:28:54.069 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39272.log.gz 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39480.log 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40990.log.gz 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40618.log 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41527.log.gz 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.39480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41625.log 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39480.log.gz 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40498.log 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40618.log.gz 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41429.log 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41625.log.gz 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40207.log 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40498.log.gz 2026-02-23T10:28:54.070 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39566.log 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41429.log.gz 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40350.log 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40207.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40207.log.gz 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.39566.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39566.log.gz 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41282.log 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40350.log.gz 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41723.log 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr: 83.0% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40378.log 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41282.log.gz 2026-02-23T10:28:54.071 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41820.log 2026-02-23T10:28:54.072 INFO:teuthology.orchestra.run.trial038.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41723.log.gz 2026-02-23T10:28:54.072 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.40378.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40378.log.gz 2026-02-23T10:28:54.072 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.41820.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41820.log.gz 2026-02-23T10:28:54.073 INFO:teuthology.orchestra.run.trial038.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-23T10:28:54.078 INFO:teuthology.orchestra.run.trial153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-23T10:28:54.080 INFO:teuthology.orchestra.run.trial153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39402.log 2026-02-23T10:28:54.080 INFO:teuthology.orchestra.run.trial153.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-23T10:28:54.080 INFO:teuthology.orchestra.run.trial153.stderr:/var/log/ceph/ceph-client.admin.39402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39348.log 2026-02-23T10:28:54.080 INFO:teuthology.orchestra.run.trial153.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39402.log.gz 2026-02-23T10:28:54.080 INFO:teuthology.orchestra.run.trial153.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39250.log 2026-02-23T10:28:54.081 INFO:teuthology.orchestra.run.trial153.stderr:/var/log/ceph/ceph-client.admin.39348.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-23T10:28:54.081 INFO:teuthology.orchestra.run.trial153.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39348.log.gz 2026-02-23T10:28:54.081 INFO:teuthology.orchestra.run.trial153.stderr:/var/log/ceph/ceph-client.admin.39250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39297.log 2026-02-23T10:28:54.081 INFO:teuthology.orchestra.run.trial153.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39250.log.gz 2026-02-23T10:28:54.081 INFO:teuthology.orchestra.run.trial153.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.39297.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39297.log.gz 2026-02-23T10:28:54.087 INFO:teuthology.orchestra.run.trial153.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-23T10:28:54.102 INFO:teuthology.orchestra.run.trial038.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-23T10:28:54.121 INFO:teuthology.orchestra.run.trial038.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-23T10:28:54.123 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-23T10:28:54.123 INFO:teuthology.orchestra.run.trial038.stderr:real 0m0.083s 2026-02-23T10:28:54.123 INFO:teuthology.orchestra.run.trial038.stderr:user 0m0.158s 2026-02-23T10:28:54.123 INFO:teuthology.orchestra.run.trial038.stderr:sys 0m0.037s 2026-02-23T10:28:54.137 INFO:teuthology.orchestra.run.trial153.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-23T10:28:54.189 INFO:teuthology.orchestra.run.trial153.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-23T10:28:54.190 INFO:teuthology.orchestra.run.trial153.stderr: 2026-02-23T10:28:54.190 INFO:teuthology.orchestra.run.trial153.stderr:real 0m0.121s 2026-02-23T10:28:54.190 INFO:teuthology.orchestra.run.trial153.stderr:user 0m0.177s 2026-02-23T10:28:54.190 INFO:teuthology.orchestra.run.trial153.stderr:sys 0m0.022s 2026-02-23T10:28:54.191 INFO:tasks.ceph:Archiving logs... 2026-02-23T10:28:54.192 DEBUG:teuthology.misc:Transferring archived files from trial038:/var/log/ceph to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial/66074/remote/trial038/log 2026-02-23T10:28:54.193 DEBUG:teuthology.orchestra.run.trial038:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-23T10:28:54.330 DEBUG:teuthology.misc:Transferring archived files from trial153:/var/log/ceph to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial/66074/remote/trial153/log 2026-02-23T10:28:54.331 DEBUG:teuthology.orchestra.run.trial153:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-23T10:28:54.430 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-23T10:28:54.584 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-23T10:28:54.595 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-23T10:28:54.595 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-23T10:28:54.597 DEBUG:teuthology.orchestra.run.trial153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-23T10:28:54.622 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-23T10:28:54.634 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-23T10:28:54.634 DEBUG:teuthology.orchestra.run.trial038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-23T10:28:54.662 DEBUG:teuthology.orchestra.run.trial153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-23T10:28:54.676 INFO:teuthology.orchestra.run.trial038.stderr:bash: line 1: ntpq: command not found 2026-02-23T10:28:54.677 INFO:teuthology.orchestra.run.trial153.stderr:bash: line 1: ntpq: command not found 2026-02-23T10:28:54.702 INFO:teuthology.orchestra.run.trial038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-23T10:28:54.702 INFO:teuthology.orchestra.run.trial038.stdout:=============================================================================== 2026-02-23T10:28:54.703 INFO:teuthology.orchestra.run.trial038.stdout:^+ 144.202.66.214.vultruser> 3 6 377 35 -140us[ -140us] +/- 32ms 2026-02-23T10:28:54.703 INFO:teuthology.orchestra.run.trial038.stdout:^- gwallaceNTPpool.com 2 6 377 34 -213us[ -213us] +/- 59ms 2026-02-23T10:28:54.703 INFO:teuthology.orchestra.run.trial038.stdout:^* ntp4.glypnod.com 2 6 377 36 +611us[ +765us] +/- 14ms 2026-02-23T10:28:54.703 INFO:teuthology.orchestra.run.trial038.stdout:^- 23-93-251-54.dedicated.s> 1 6 377 34 -597us[ -597us] +/- 136ms 2026-02-23T10:28:54.708 INFO:teuthology.orchestra.run.trial153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-23T10:28:54.708 INFO:teuthology.orchestra.run.trial153.stdout:=============================================================================== 2026-02-23T10:28:54.709 INFO:teuthology.orchestra.run.trial153.stdout:^- ip216-250-115-174.pbiaas> 2 6 377 37 +294us[ +283us] +/- 106ms 2026-02-23T10:28:54.709 INFO:teuthology.orchestra.run.trial153.stdout:^- 23.186.168.130 2 6 357 93 +313us[ +304us] +/- 55ms 2026-02-23T10:28:54.709 INFO:teuthology.orchestra.run.trial153.stdout:^- 234.201.68.199.in-addr.a> 2 6 377 36 -221us[ -232us] +/- 51ms 2026-02-23T10:28:54.709 INFO:teuthology.orchestra.run.trial153.stdout:^* chi3.us.ntp.li 2 6 377 35 -25us[ -36us] +/- 11ms 2026-02-23T10:28:54.710 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-23T10:28:54.723 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-23T10:28:54.724 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-23T10:28:54.740 DEBUG:teuthology.orchestra.run.trial038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-23T10:28:54.799 DEBUG:teuthology.orchestra.run.trial153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-23T10:28:54.854 DEBUG:teuthology.orchestra.run.trial038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-23T10:28:54.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T10:28:54.884 DEBUG:teuthology.orchestra.run.trial153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-23T10:28:54.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T10:28:54.911 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-23T10:28:54.927 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-23T10:28:54.942 INFO:teuthology.task.internal:Duration was 660.053479 seconds 2026-02-23T10:28:54.943 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-23T10:28:54.958 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-23T10:28:54.958 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-23T10:28:54.961 DEBUG:teuthology.orchestra.run.trial153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-23T10:28:54.997 INFO:teuthology.orchestra.run.trial038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-23T10:28:54.999 INFO:teuthology.orchestra.run.trial153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-23T10:28:55.377 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-23T10:28:55.377 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial038.front.sepia.ceph.com 2026-02-23T10:28:55.377 DEBUG:teuthology.orchestra.run.trial038:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-23T10:28:55.436 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial153.front.sepia.ceph.com 2026-02-23T10:28:55.437 DEBUG:teuthology.orchestra.run.trial153:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-23T10:28:55.457 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-23T10:28:55.458 DEBUG:teuthology.orchestra.run.trial038:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-23T10:28:55.479 DEBUG:teuthology.orchestra.run.trial153:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-23T10:28:55.558 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-23T10:28:55.559 DEBUG:teuthology.orchestra.run.trial038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-23T10:28:55.561 DEBUG:teuthology.orchestra.run.trial153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-23T10:28:55.582 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-23T10:28:55.583 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-23T10:28:55.583 INFO:teuthology.orchestra.run.trial038.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-23T10:28:55.583 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-23T10:28:55.583 INFO:teuthology.orchestra.run.trial038.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-23T10:28:55.584 INFO:teuthology.orchestra.run.trial153.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-23T10:28:55.585 INFO:teuthology.orchestra.run.trial153.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-23T10:28:55.585 INFO:teuthology.orchestra.run.trial153.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-23T10:28:55.586 INFO:teuthology.orchestra.run.trial153.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-23T10:28:55.586 INFO:teuthology.orchestra.run.trial153.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-23T10:28:55.592 INFO:teuthology.orchestra.run.trial038.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-23T10:28:55.595 INFO:teuthology.orchestra.run.trial153.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-23T10:28:55.597 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-23T10:28:55.615 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-23T10:28:55.615 DEBUG:teuthology.orchestra.run.trial038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-23T10:28:55.659 DEBUG:teuthology.orchestra.run.trial153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-23T10:28:55.683 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-23T10:28:55.700 DEBUG:teuthology.orchestra.run.trial038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-23T10:28:55.703 DEBUG:teuthology.orchestra.run.trial153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-23T10:28:55.723 INFO:teuthology.orchestra.run.trial038.stdout:kernel.core_pattern = core 2026-02-23T10:28:55.748 INFO:teuthology.orchestra.run.trial153.stdout:kernel.core_pattern = core 2026-02-23T10:28:55.759 DEBUG:teuthology.orchestra.run.trial038:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-23T10:28:55.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T10:28:55.792 DEBUG:teuthology.orchestra.run.trial153:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-23T10:28:55.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T10:28:55.816 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-23T10:28:55.831 INFO:teuthology.task.internal:Transferring archived files... 2026-02-23T10:28:55.832 DEBUG:teuthology.misc:Transferring archived files from trial038:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial/66074/remote/trial038 2026-02-23T10:28:55.832 DEBUG:teuthology.orchestra.run.trial038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-23T10:28:55.874 DEBUG:teuthology.misc:Transferring archived files from trial153:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_09:58:48-crimson-rados-main-distro-debug-trial/66074/remote/trial153 2026-02-23T10:28:55.874 DEBUG:teuthology.orchestra.run.trial153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-23T10:28:55.915 INFO:teuthology.task.internal:Removing archive directory... 2026-02-23T10:28:55.915 DEBUG:teuthology.orchestra.run.trial038:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-23T10:28:55.918 DEBUG:teuthology.orchestra.run.trial153:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-23T10:28:55.960 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-23T10:28:55.977 INFO:teuthology.task.internal:Not uploading archives. 2026-02-23T10:28:55.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-23T10:28:55.996 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-23T10:28:55.996 DEBUG:teuthology.orchestra.run.trial038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-23T10:28:55.999 DEBUG:teuthology.orchestra.run.trial153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-23T10:28:56.014 INFO:teuthology.orchestra.run.trial038.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 23 10:28 /home/ubuntu/cephtest 2026-02-23T10:28:56.015 INFO:teuthology.orchestra.run.trial153.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 23 10:28 /home/ubuntu/cephtest 2026-02-23T10:28:56.017 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-23T10:28:56.035 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-23T10:28:56.077 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/cls} duration: 660.053478717804 flavor: debug owner: scheduled_shraddhaag@soko04.front.sepia.ceph.com success: true 2026-02-23T10:28:56.077 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-23T10:28:56.192 INFO:teuthology.run:pass