2026-02-21T14:28:35.554 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-21T14:28:35.570 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T14:28:35.648 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/xuxuehan-2026-02-21_14:23:14-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62724 branch: wip-trans-cc-opt-rbm-drop-brf description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} tasks/rbd_lock_and_fence} email: xxhdx1985126@gmail.com first_in_suite: false flavor: debug job_id: '62724' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: xuxuehan-2026-02-21_14:23:14-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-trans-cc-opt-rbm-drop-brf ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug osd: 20 debug seastore: 25 debug seastore backref: 20 debug seastore cache: 25 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 25 debug seastore odata: 25 debug seastore omap: 25 debug seastore onode: 25 debug seastore t: 25 debug seastore tm: 25 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) sha1: 57019f91dea307b3c704ce2392a5f854512b5900 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: 57019f91dea307b3c704ce2392a5f854512b5900 workunit: branch: wip-seastore-qa-log-level-up env: CRIMSON_COMPAT: '1' sha1: c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f owner: scheduled_xuxuehan@soko04.front.sepia.ceph.com priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 2477 sha1: 57019f91dea307b3c704ce2392a5f854512b5900 sleep_before_teardown: 0 suite: crimson-rados suite_branch: wip-seastore-qa-log-level-up suite_path: /home/teuthworker/src/github.com_xxhdx1985126_ceph_c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f/qa suite_relpath: qa suite_repo: https://github.com/xxhdx1985126/ceph.git suite_sha1: c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f targets: trial130.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGZX6TDggcgyxvoWI80vWWs3SiZGCI2rO9OX45Gc0UEkzgZVmZXcqujSs4uc/rIhUkF9dqQg6aUenr/qMTo4jwM= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 - workunit: clients: client.0: - rbd/test_lock_fence.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-21_14:23:14 tube: trial user: xuxuehan verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-21T14:28:35.648 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_xxhdx1985126_ceph_c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f/qa; will attempt to use it 2026-02-21T14:28:35.648 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_xxhdx1985126_ceph_c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f/qa/tasks 2026-02-21T14:28:35.648 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-21T14:28:35.650 INFO:teuthology.task.internal:Checking packages... 2026-02-21T14:28:35.650 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '57019f91dea307b3c704ce2392a5f854512b5900' 2026-02-21T14:28:35.650 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-21T14:28:35.650 INFO:teuthology.packaging:ref: None 2026-02-21T14:28:35.650 INFO:teuthology.packaging:tag: None 2026-02-21T14:28:35.650 INFO:teuthology.packaging:branch: wip-trans-cc-opt-rbm-drop-brf 2026-02-21T14:28:35.650 INFO:teuthology.packaging:sha1: 57019f91dea307b3c704ce2392a5f854512b5900 2026-02-21T14:28:35.650 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-trans-cc-opt-rbm-drop-brf 2026-02-21T14:28:35.773 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5076.g57019f91 2026-02-21T14:28:35.775 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-21T14:28:35.789 INFO:teuthology.task.internal:no buildpackages task found 2026-02-21T14:28:35.789 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-21T14:28:35.802 INFO:teuthology.task.internal:Saving configuration 2026-02-21T14:28:35.811 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-21T14:28:35.824 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-21T14:28:35.884 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial130.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/xuxuehan-2026-02-21_14:23:14-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62724', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-21 14:23:49.641618', 'locked_by': 'scheduled_xuxuehan@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGZX6TDggcgyxvoWI80vWWs3SiZGCI2rO9OX45Gc0UEkzgZVmZXcqujSs4uc/rIhUkF9dqQg6aUenr/qMTo4jwM='} 2026-02-21T14:28:35.884 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-21T14:28:35.895 INFO:teuthology.task.internal:roles: ubuntu@trial130.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-21T14:28:35.895 INFO:teuthology.run_tasks:Running task console_log... 2026-02-21T14:28:35.969 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7e93302d34c0>, signals=[15]) 2026-02-21T14:28:35.970 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-21T14:28:35.981 INFO:teuthology.task.internal:Opening connections... 2026-02-21T14:28:35.981 DEBUG:teuthology.task.internal:connecting to ubuntu@trial130.front.sepia.ceph.com 2026-02-21T14:28:35.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T14:28:36.042 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-21T14:28:36.052 DEBUG:teuthology.orchestra.run.trial130:> uname -m 2026-02-21T14:28:36.125 INFO:teuthology.orchestra.run.trial130.stdout:x86_64 2026-02-21T14:28:36.125 DEBUG:teuthology.orchestra.run.trial130:> cat /etc/os-release 2026-02-21T14:28:36.181 INFO:teuthology.orchestra.run.trial130.stdout:NAME="CentOS Stream" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:VERSION="9" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:ID="centos" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:ID_LIKE="rhel fedora" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:VERSION_ID="9" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:PLATFORM_ID="platform:el9" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:ANSI_COLOR="0;31" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:LOGO="fedora-logo-icon" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:HOME_URL="https://centos.org/" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-21T14:28:36.182 INFO:teuthology.orchestra.run.trial130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-21T14:28:36.182 INFO:teuthology.lock.ops:Updating trial130.front.sepia.ceph.com on lock server 2026-02-21T14:28:36.240 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-21T14:28:36.255 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-21T14:28:36.271 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-21T14:28:36.271 DEBUG:teuthology.orchestra.run.trial130:> test '!' -e /home/ubuntu/cephtest 2026-02-21T14:28:36.286 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-21T14:28:36.300 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-21T14:28:36.301 DEBUG:teuthology.orchestra.run.trial130:> test -z $(ls -A /var/lib/ceph) 2026-02-21T14:28:36.343 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-21T14:28:36.411 INFO:teuthology.run_tasks:Running task kernel... 2026-02-21T14:28:36.432 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-21T14:28:36.432 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-21T14:28:36.432 DEBUG:teuthology.orchestra.run.trial130:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-21T14:28:36.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T14:28:36.448 DEBUG:teuthology.orchestra.run.trial130:> uname -r 2026-02-21T14:28:36.504 INFO:teuthology.orchestra.run.trial130.stdout:5.14.0-681.el9.x86_64 2026-02-21T14:28:36.504 INFO:teuthology.task.kernel:Running kernel on trial130: 5.14.0-681.el9.x86_64 2026-02-21T14:28:36.504 DEBUG:teuthology.orchestra.run.trial130:> sudo yum install -y kernel 2026-02-21T14:28:38.686 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.9 MB 00:01 2026-02-21T14:28:42.410 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 27 MB 00:03 2026-02-21T14:28:45.987 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 7.8 MB 00:01 2026-02-21T14:28:46.813 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - Extras packages 230 kB/s | 20 kB 00:00 2026-02-21T14:28:47.619 INFO:teuthology.orchestra.run.trial130.stdout:Extra Packages for Enterprise Linux 27 MB/s | 20 MB 00:00 2026-02-21T14:28:50.786 INFO:teuthology.orchestra.run.trial130.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-21T14:28:51.787 INFO:teuthology.orchestra.run.trial130.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-21T14:28:51.787 INFO:teuthology.orchestra.run.trial130.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-21T14:28:51.806 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:28:51.806 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:28:51.806 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:28:51.830 DEBUG:teuthology.orchestra.run.trial130:> echo no | sudo yum reinstall kernel || true 2026-02-21T14:28:52.126 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 02:28:50 PM UTC. 2026-02-21T14:28:52.206 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:28:52.206 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:28:52.206 INFO:teuthology.orchestra.run.trial130.stdout: Package Architecture Version Repository Size 2026-02-21T14:28:52.207 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:28:52.207 INFO:teuthology.orchestra.run.trial130.stdout:Reinstalling: 2026-02-21T14:28:52.207 INFO:teuthology.orchestra.run.trial130.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-21T14:28:52.207 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:28:52.207 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:28:52.207 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:28:52.207 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:28:52.207 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 803 k 2026-02-21T14:28:52.207 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 0 2026-02-21T14:28:52.207 INFO:teuthology.orchestra.run.trial130.stderr:Operation aborted. 2026-02-21T14:28:52.227 INFO:teuthology.orchestra.run.trial130.stdout:Is this ok [y/N]: 2026-02-21T14:28:52.227 DEBUG:teuthology.orchestra.run.trial130:> sudo yum reinstall -y kernel || true 2026-02-21T14:28:52.526 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 02:28:50 PM UTC. 2026-02-21T14:28:52.604 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout: Package Architecture Version Repository Size 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout:Reinstalling: 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 803 k 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 0 2026-02-21T14:28:52.605 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:28:52.822 INFO:teuthology.orchestra.run.trial130.stdout:kernel-5.14.0-681.el9.x86_64.rpm 5.1 MB/s | 803 kB 00:00 2026-02-21T14:28:52.822 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:28:52.822 INFO:teuthology.orchestra.run.trial130.stdout:Total 3.6 MB/s | 803 kB 00:00 2026-02-21T14:28:52.832 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:28:52.865 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:28:52.865 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:28:52.870 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:28:52.870 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:28:52.893 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:28:52.897 INFO:teuthology.orchestra.run.trial130.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-21T14:28:52.897 INFO:teuthology.orchestra.run.trial130.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T14:28:53.009 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T14:28:53.009 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-21T14:28:53.062 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T14:28:53.063 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:28:53.063 INFO:teuthology.orchestra.run.trial130.stdout:Reinstalled: 2026-02-21T14:28:53.063 INFO:teuthology.orchestra.run.trial130.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-21T14:28:53.063 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:28:53.063 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:28:53.091 DEBUG:teuthology.orchestra.run.trial130:> rpm -q kernel | sort -rV | head -n 1 2026-02-21T14:28:53.116 INFO:teuthology.orchestra.run.trial130.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-21T14:28:53.116 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-21T14:28:53.116 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-21T14:28:53.116 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-21T14:28:53.116 DEBUG:teuthology.orchestra.run.trial130:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-21T14:28:53.179 INFO:teuthology.orchestra.run.trial130.stdout:ttyS1 2026-02-21T14:28:53.203 DEBUG:teuthology.parallel:result is None 2026-02-21T14:28:53.203 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-21T14:28:53.215 INFO:teuthology.task.internal:Creating test directory... 2026-02-21T14:28:53.215 DEBUG:teuthology.orchestra.run.trial130:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-21T14:28:53.259 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-21T14:28:53.271 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-21T14:28:53.285 INFO:teuthology.task.internal:Creating archive directory... 2026-02-21T14:28:53.285 DEBUG:teuthology.orchestra.run.trial130:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-21T14:28:53.317 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-21T14:28:53.330 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-21T14:28:53.330 DEBUG:teuthology.orchestra.run.trial130:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-21T14:28:53.380 INFO:teuthology.orchestra.run.trial130.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T14:28:53.389 INFO:teuthology.orchestra.run.trial130.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T14:28:53.390 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-21T14:28:53.402 INFO:teuthology.task.internal:Configuring sudo... 2026-02-21T14:28:53.402 DEBUG:teuthology.orchestra.run.trial130:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-21T14:28:53.454 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-21T14:28:53.469 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-21T14:28:53.469 DEBUG:teuthology.orchestra.run.trial130:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-21T14:28:53.511 DEBUG:teuthology.orchestra.run.trial130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T14:28:53.635 DEBUG:teuthology.orchestra.run.trial130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T14:28:53.659 DEBUG:teuthology.orchestra.run.trial130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T14:28:53.795 DEBUG:teuthology.orchestra.run.trial130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T14:28:53.817 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:28:53.817 DEBUG:teuthology.orchestra.run.trial130:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-21T14:28:53.880 DEBUG:teuthology.orchestra.run.trial130:> sudo service rsyslog restart 2026-02-21T14:28:53.947 INFO:teuthology.orchestra.run.trial130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-21T14:28:54.437 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-21T14:28:54.448 INFO:teuthology.task.internal:Starting timer... 2026-02-21T14:28:54.448 INFO:teuthology.run_tasks:Running task pcp... 2026-02-21T14:28:54.467 INFO:teuthology.run_tasks:Running task selinux... 2026-02-21T14:28:54.484 DEBUG:teuthology.orchestra.run.trial130:> sudo service auditd rotate 2026-02-21T14:28:54.513 INFO:teuthology.orchestra.run.trial130.stdout:Rotating logs: 2026-02-21T14:28:54.515 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-21T14:28:54.515 DEBUG:teuthology.orchestra.run.trial130:> /usr/sbin/getenforce 2026-02-21T14:28:54.571 INFO:teuthology.orchestra.run.trial130.stdout:Permissive 2026-02-21T14:28:54.571 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial130.front.sepia.ceph.com': 'permissive'} 2026-02-21T14:28:54.571 DEBUG:teuthology.orchestra.run.trial130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-21T14:28:54.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T14:28:54.636 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-21T14:28:54.636 DEBUG:teuthology.orchestra.run.trial130:> sudo /usr/sbin/setenforce permissive 2026-02-21T14:28:54.701 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-21T14:28:54.712 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-21T14:28:54.719 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-21T14:28:54.720 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial130.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-21T14:31:16.497 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial130.front.sepia.ceph.com')] 2026-02-21T14:31:16.498 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial130.front.sepia.ceph.com' 2026-02-21T14:31:16.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T14:31:16.567 DEBUG:teuthology.orchestra.run.trial130:> true 2026-02-21T14:31:16.651 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial130.front.sepia.ceph.com' 2026-02-21T14:31:16.651 INFO:teuthology.run_tasks:Running task clock... 2026-02-21T14:31:16.665 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-21T14:31:16.665 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-21T14:31:16.665 DEBUG:teuthology.orchestra.run.trial130:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T14:31:16.717 INFO:teuthology.orchestra.run.trial130.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-21T14:31:16.728 INFO:teuthology.orchestra.run.trial130.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-21T14:31:16.749 INFO:teuthology.orchestra.run.trial130.stderr:sudo: ntpd: command not found 2026-02-21T14:31:16.759 INFO:teuthology.orchestra.run.trial130.stdout:506 Cannot talk to daemon 2026-02-21T14:31:16.771 INFO:teuthology.orchestra.run.trial130.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-21T14:31:16.782 INFO:teuthology.orchestra.run.trial130.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-21T14:31:16.833 INFO:teuthology.orchestra.run.trial130.stderr:bash: line 1: ntpq: command not found 2026-02-21T14:31:16.842 INFO:teuthology.orchestra.run.trial130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-21T14:31:16.842 INFO:teuthology.orchestra.run.trial130.stdout:=============================================================================== 2026-02-21T14:31:16.842 INFO:teuthology.orchestra.run.trial130.stdout:^? 173.208.172.164 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T14:31:16.842 INFO:teuthology.orchestra.run.trial130.stdout:^? chi3.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T14:31:16.842 INFO:teuthology.orchestra.run.trial130.stdout:^? ntp2.wiktel.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T14:31:16.842 INFO:teuthology.orchestra.run.trial130.stdout:^? pool-138-89-14-60.mad.ea> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T14:31:16.844 INFO:teuthology.run_tasks:Running task install... 2026-02-21T14:31:16.863 DEBUG:teuthology.task.install:project ceph 2026-02-21T14:31:16.863 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '57019f91dea307b3c704ce2392a5f854512b5900'}} 2026-02-21T14:31:16.863 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '57019f91dea307b3c704ce2392a5f854512b5900'} 2026-02-21T14:31:16.863 INFO:teuthology.task.install:Using flavor: debug 2026-02-21T14:31:16.873 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T14:31:16.873 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-21T14:31:16.874 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '57019f91dea307b3c704ce2392a5f854512b5900', 'tag': None, 'wait_for_package': False} 2026-02-21T14:31:16.875 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=57019f91dea307b3c704ce2392a5f854512b5900 2026-02-21T14:31:17.043 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-trans-cc-opt-rbm-drop-brf/57019f91dea307b3c704ce2392a5f854512b5900/centos/9/flavors/debug/ 2026-02-21T14:31:17.043 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5076.g57019f91 2026-02-21T14:31:17.423 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-trans-cc-opt-rbm-drop-brf/57019f91dea307b3c704ce2392a5f854512b5900/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-trans-cc-opt-rbm-drop-brf/57019f91dea307b3c704ce2392a5f854512b5900/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-trans-cc-opt-rbm-drop-brf/57019f91dea307b3c704ce2392a5f854512b5900/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-21T14:31:17.423 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:31:17.423 DEBUG:teuthology.orchestra.run.trial130:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-21T14:31:17.449 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-21T14:31:17.450 DEBUG:teuthology.orchestra.run.trial130:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/57019f91dea307b3c704ce2392a5f854512b5900/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-21T14:31:17.514 DEBUG:teuthology.orchestra.run.trial130:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-21T14:31:17.586 DEBUG:teuthology.orchestra.run.trial130:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-21T14:31:17.651 INFO:teuthology.orchestra.run.trial130.stdout:check_obsoletes = 1 2026-02-21T14:31:17.652 DEBUG:teuthology.orchestra.run.trial130:> sudo yum clean all 2026-02-21T14:31:17.812 INFO:teuthology.orchestra.run.trial130.stdout:41 files removed 2026-02-21T14:31:17.826 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-radosgw 2026-02-21T14:31:18.674 INFO:teuthology.orchestra.run.trial130.stdout:ceph packages for x86_64 133 kB/s | 94 kB 00:00 2026-02-21T14:31:19.338 INFO:teuthology.orchestra.run.trial130.stdout:ceph noarch packages 38 kB/s | 24 kB 00:00 2026-02-21T14:31:19.933 INFO:teuthology.orchestra.run.trial130.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-21T14:31:20.823 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.9 MB 00:00 2026-02-21T14:31:23.387 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - AppStream 13 MB/s | 27 MB 00:02 2026-02-21T14:31:27.621 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 7.8 MB 00:01 2026-02-21T14:31:28.484 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - Extras packages 144 kB/s | 20 kB 00:00 2026-02-21T14:31:28.961 INFO:teuthology.orchestra.run.trial130.stdout:Extra Packages for Enterprise Linux 50 MB/s | 20 MB 00:00 2026-02-21T14:31:32.179 INFO:teuthology.orchestra.run.trial130.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-21T14:31:33.376 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:31:33.377 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:31:33.377 INFO:teuthology.orchestra.run.trial130.stdout: Package Arch Version Repository Size 2026-02-21T14:31:33.377 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:31:33.377 INFO:teuthology.orchestra.run.trial130.stdout:Installing: 2026-02-21T14:31:33.377 INFO:teuthology.orchestra.run.trial130.stdout: ceph-radosgw x86_64 2:20.3.0-5076.g57019f91.el9 ceph 11 M 2026-02-21T14:31:33.377 INFO:teuthology.orchestra.run.trial130.stdout:Upgrading: 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout: librados2 x86_64 2:20.3.0-5076.g57019f91.el9 ceph 3.3 M 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout: librbd1 x86_64 2:20.3.0-5076.g57019f91.el9 ceph 2.7 M 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout:Installing dependencies: 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout: ceph-base x86_64 2:20.3.0-5076.g57019f91.el9 ceph 5.1 M 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout: ceph-common x86_64 2:20.3.0-5076.g57019f91.el9 ceph 21 M 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout: ceph-selinux x86_64 2:20.3.0-5076.g57019f91.el9 ceph 25 k 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout: libcephfs2 x86_64 2:20.3.0-5076.g57019f91.el9 ceph 859 k 2026-02-21T14:31:33.378 INFO:teuthology.orchestra.run.trial130.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: libradosstriper1 x86_64 2:20.3.0-5076.g57019f91.el9 ceph 263 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: librgw2 x86_64 2:20.3.0-5076.g57019f91.el9 ceph 5.6 M 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: python3-ceph-argparse x86_64 2:20.3.0-5076.g57019f91.el9 ceph 45 k 2026-02-21T14:31:33.379 INFO:teuthology.orchestra.run.trial130.stdout: python3-ceph-common x86_64 2:20.3.0-5076.g57019f91.el9 ceph 184 k 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout: python3-cephfs x86_64 2:20.3.0-5076.g57019f91.el9 ceph 204 k 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout: python3-rados x86_64 2:20.3.0-5076.g57019f91.el9 ceph 324 k 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout: python3-rbd x86_64 2:20.3.0-5076.g57019f91.el9 ceph 305 k 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout: python3-rgw x86_64 2:20.3.0-5076.g57019f91.el9 ceph 100 k 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout:Installing weak dependencies: 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout: libcephfs-daemon x86_64 2:20.3.0-5076.g57019f91.el9 ceph 35 k 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5076.g57019f91.el9 ceph 24 k 2026-02-21T14:31:33.380 INFO:teuthology.orchestra.run.trial130.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-21T14:31:33.381 INFO:teuthology.orchestra.run.trial130.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-21T14:31:33.381 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:31:33.381 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:31:33.381 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:31:33.381 INFO:teuthology.orchestra.run.trial130.stdout:Install 35 Packages 2026-02-21T14:31:33.381 INFO:teuthology.orchestra.run.trial130.stdout:Upgrade 2 Packages 2026-02-21T14:31:33.381 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:31:33.382 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 61 M 2026-02-21T14:31:33.382 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:31:36.410 INFO:teuthology.orchestra.run.trial130.stdout:(1/37): ceph-base-20.3.0-5076.g57019f91.el9.x86 1.8 MB/s | 5.1 MB 00:02 2026-02-21T14:31:36.481 INFO:teuthology.orchestra.run.trial130.stdout:(2/37): ceph-selinux-20.3.0-5076.g57019f91.el9. 352 kB/s | 25 kB 00:00 2026-02-21T14:31:36.553 INFO:teuthology.orchestra.run.trial130.stdout:(3/37): libcephfs-daemon-20.3.0-5076.g57019f91. 488 kB/s | 35 kB 00:00 2026-02-21T14:31:36.625 INFO:teuthology.orchestra.run.trial130.stdout:(4/37): libcephfs-proxy2-20.3.0-5076.g57019f91. 336 kB/s | 24 kB 00:00 2026-02-21T14:31:37.245 INFO:teuthology.orchestra.run.trial130.stdout:(5/37): libcephfs2-20.3.0-5076.g57019f91.el9.x8 1.4 MB/s | 859 kB 00:00 2026-02-21T14:31:37.452 INFO:teuthology.orchestra.run.trial130.stdout:(6/37): libradosstriper1-20.3.0-5076.g57019f91. 1.2 MB/s | 263 kB 00:00 2026-02-21T14:31:54.532 INFO:teuthology.orchestra.run.trial130.stdout:(7/37): ceph-radosgw-20.3.0-5076.g57019f91.el9. 547 kB/s | 11 MB 00:20 2026-02-21T14:31:54.671 INFO:teuthology.orchestra.run.trial130.stdout:(8/37): python3-ceph-argparse-20.3.0-5076.g5701 324 kB/s | 45 kB 00:00 2026-02-21T14:31:55.156 INFO:teuthology.orchestra.run.trial130.stdout:(9/37): python3-ceph-common-20.3.0-5076.g57019f 380 kB/s | 184 kB 00:00 2026-02-21T14:31:55.640 INFO:teuthology.orchestra.run.trial130.stdout:(10/37): python3-cephfs-20.3.0-5076.g57019f91.e 421 kB/s | 204 kB 00:00 2026-02-21T14:31:56.236 INFO:teuthology.orchestra.run.trial130.stdout:(11/37): librgw2-20.3.0-5076.g57019f91.el9.x86_ 307 kB/s | 5.6 MB 00:18 2026-02-21T14:31:56.268 INFO:teuthology.orchestra.run.trial130.stdout:(12/37): python3-rados-20.3.0-5076.g57019f91.el 515 kB/s | 324 kB 00:00 2026-02-21T14:31:56.478 INFO:teuthology.orchestra.run.trial130.stdout:(13/37): python3-rgw-20.3.0-5076.g57019f91.el9. 478 kB/s | 100 kB 00:00 2026-02-21T14:31:57.037 INFO:teuthology.orchestra.run.trial130.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 59 kB/s | 33 kB 00:00 2026-02-21T14:31:57.346 INFO:teuthology.orchestra.run.trial130.stdout:(15/37): python3-rbd-20.3.0-5076.g57019f91.el9. 275 kB/s | 305 kB 00:01 2026-02-21T14:31:57.511 INFO:teuthology.orchestra.run.trial130.stdout:(16/37): nvme-cli-2.16-1.el9.x86_64.rpm 2.4 MB/s | 1.2 MB 00:00 2026-02-21T14:31:57.670 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] boost-program-options-1.75.0-13.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-13.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:31:57.748 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] pciutils-3.7.0-7.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/pciutils-3.7.0-7.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:31:58.319 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] boost-program-options-1.75.0-13.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-13.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:31:58.359 INFO:teuthology.orchestra.run.trial130.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 123 kB/s | 104 kB 00:00 2026-02-21T14:31:58.549 INFO:teuthology.orchestra.run.trial130.stdout:(18/37): pciutils-3.7.0-7.el9.x86_64.rpm 77 kB/s | 93 kB 00:01 2026-02-21T14:31:58.708 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] libpmemobj-1.12.1-1.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libpmemobj-1.12.1-1.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:31:59.074 INFO:teuthology.orchestra.run.trial130.stdout:(19/37): libnbd-1.20.3-4.el9.x86_64.rpm 229 kB/s | 164 kB 00:00 2026-02-21T14:31:59.232 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] librabbitmq-0.11.0-7.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librabbitmq-0.11.0-7.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:31:59.360 INFO:teuthology.orchestra.run.trial130.stdout:(20/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 198 kB/s | 160 kB 00:00 2026-02-21T14:31:59.519 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] librdkafka-1.6.1-102.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librdkafka-1.6.1-102.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:31:59.785 INFO:teuthology.orchestra.run.trial130.stdout:(21/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 64 kB/s | 45 kB 00:00 2026-02-21T14:31:59.931 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] librdkafka-1.6.1-102.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librdkafka-1.6.1-102.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:31:59.946 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] lttng-ust-2.12.0-6.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/lttng-ust-2.12.0-6.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:31:59.990 INFO:teuthology.orchestra.run.trial130.stdout:(22/37): librdkafka-1.6.1-102.el9.x86_64.rpm 1.0 MB/s | 662 kB 00:00 2026-02-21T14:32:00.003 INFO:teuthology.orchestra.run.trial130.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 18 MB/s | 240 kB 00:00 2026-02-21T14:32:00.007 INFO:teuthology.orchestra.run.trial130.stdout:(24/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.3 MB/s | 292 kB 00:00 2026-02-21T14:32:00.013 INFO:teuthology.orchestra.run.trial130.stdout:(25/37): qatlib-service-25.08.0-2.el9.x86_64.rp 3.7 MB/s | 37 kB 00:00 2026-02-21T14:32:00.018 INFO:teuthology.orchestra.run.trial130.stdout:(26/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 5.9 MB/s | 66 kB 00:00 2026-02-21T14:32:00.242 INFO:teuthology.orchestra.run.trial130.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.3 MB/s | 308 kB 00:00 2026-02-21T14:32:00.275 INFO:teuthology.orchestra.run.trial130.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 748 kB/s | 25 kB 00:00 2026-02-21T14:32:00.309 INFO:teuthology.orchestra.run.trial130.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2026-02-21T14:32:00.345 INFO:teuthology.orchestra.run.trial130.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 1.9 MB/s | 67 kB 00:00 2026-02-21T14:32:00.429 INFO:teuthology.orchestra.run.trial130.stdout:(31/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 9.7 MB/s | 838 kB 00:00 2026-02-21T14:32:00.435 INFO:teuthology.orchestra.run.trial130.stdout:(32/37): libarrow-9.0.0-15.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2026-02-21T14:32:00.469 INFO:teuthology.orchestra.run.trial130.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 4.7 MB/s | 191 kB 00:00 2026-02-21T14:32:00.552 INFO:teuthology.orchestra.run.trial130.stdout:(34/37): thrift-0.15.0-4.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2026-02-21T14:32:08.261 INFO:teuthology.orchestra.run.trial130.stdout:(35/37): librbd1-20.3.0-5076.g57019f91.el9.x86_ 358 kB/s | 2.7 MB 00:07 2026-02-21T14:32:09.667 INFO:teuthology.orchestra.run.trial130.stdout:(36/37): librados2-20.3.0-5076.g57019f91.el9.x8 366 kB/s | 3.3 MB 00:09 2026-02-21T14:32:25.147 INFO:teuthology.orchestra.run.trial130.stdout:(37/37): ceph-common-20.3.0-5076.g57019f91.el9. 413 kB/s | 21 MB 00:51 2026-02-21T14:32:25.151 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:32:25.151 INFO:teuthology.orchestra.run.trial130.stdout:Total 1.2 MB/s | 61 MB 00:51 2026-02-21T14:32:25.240 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:32:25.258 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:32:25.258 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:32:25.511 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:32:25.512 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:32:25.803 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:32:25.942 INFO:teuthology.orchestra.run.trial130.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-21T14:32:25.946 INFO:teuthology.orchestra.run.trial130.stdout: Upgrading : librados2-2:20.3.0-5076.g57019f91.el9.x86_64 2/39 2026-02-21T14:32:25.996 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: librados2-2:20.3.0-5076.g57019f91.el9.x86_64 2/39 2026-02-21T14:32:25.997 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libcephfs2-2:20.3.0-5076.g57019f91.el9.x86_64 3/39 2026-02-21T14:32:26.009 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: libcephfs2-2:20.3.0-5076.g57019f91.el9.x86_64 3/39 2026-02-21T14:32:26.011 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libcephfs-daemon-2:20.3.0-5076.g57019f91.el9.x86_6 4/39 2026-02-21T14:32:26.012 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libcephfs-proxy2-2:20.3.0-5076.g57019f91.el9.x86_6 5/39 2026-02-21T14:32:26.039 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5076.g57019f91.el9.x86_6 5/39 2026-02-21T14:32:26.042 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-rados-2:20.3.0-5076.g57019f91.el9.x86_64 6/39 2026-02-21T14:32:26.051 INFO:teuthology.orchestra.run.trial130.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-21T14:32:26.053 INFO:teuthology.orchestra.run.trial130.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-21T14:32:26.056 INFO:teuthology.orchestra.run.trial130.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-21T14:32:26.060 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-21T14:32:26.072 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-ceph-argparse-2:20.3.0-5076.g57019f91.el9. 11/39 2026-02-21T14:32:26.085 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-cephfs-2:20.3.0-5076.g57019f91.el9.x86_64 12/39 2026-02-21T14:32:26.086 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libradosstriper1-2:20.3.0-5076.g57019f91.el9.x86_6 13/39 2026-02-21T14:32:26.107 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5076.g57019f91.el9.x86_6 13/39 2026-02-21T14:32:26.111 INFO:teuthology.orchestra.run.trial130.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-21T14:32:26.142 INFO:teuthology.orchestra.run.trial130.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-21T14:32:26.147 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-21T14:32:26.153 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-21T14:32:26.156 INFO:teuthology.orchestra.run.trial130.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-21T14:32:26.161 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-21T14:32:26.282 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-21T14:32:26.283 INFO:teuthology.orchestra.run.trial130.stdout: Upgrading : librbd1-2:20.3.0-5076.g57019f91.el9.x86_64 21/39 2026-02-21T14:32:26.309 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: librbd1-2:20.3.0-5076.g57019f91.el9.x86_64 21/39 2026-02-21T14:32:26.311 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-rbd-2:20.3.0-5076.g57019f91.el9.x86_64 22/39 2026-02-21T14:32:26.317 INFO:teuthology.orchestra.run.trial130.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-21T14:32:26.579 INFO:teuthology.orchestra.run.trial130.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-21T14:32:26.581 INFO:teuthology.orchestra.run.trial130.stdout: Installing : librgw2-2:20.3.0-5076.g57019f91.el9.x86_64 25/39 2026-02-21T14:32:26.597 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: librgw2-2:20.3.0-5076.g57019f91.el9.x86_64 25/39 2026-02-21T14:32:26.603 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-rgw-2:20.3.0-5076.g57019f91.el9.x86_64 26/39 2026-02-21T14:32:26.607 INFO:teuthology.orchestra.run.trial130.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-21T14:32:26.610 INFO:teuthology.orchestra.run.trial130.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-21T14:32:26.610 INFO:teuthology.orchestra.run.trial130.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-21T14:32:26.626 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-21T14:32:26.742 INFO:teuthology.orchestra.run.trial130.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-21T14:32:26.744 INFO:teuthology.orchestra.run.trial130.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-21T14:32:27.024 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-21T14:32:27.024 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-21T14:32:27.024 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:32:27.067 INFO:teuthology.orchestra.run.trial130.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-21T14:32:27.069 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-ceph-common-2:20.3.0-5076.g57019f91.el9.x8 33/39 2026-02-21T14:32:28.012 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-common-2:20.3.0-5076.g57019f91.el9.x86_64 34/39 2026-02-21T14:32:28.014 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-common-2:20.3.0-5076.g57019f91.el9.x86_64 34/39 2026-02-21T14:32:28.273 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-common-2:20.3.0-5076.g57019f91.el9.x86_64 34/39 2026-02-21T14:32:28.277 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-base-2:20.3.0-5076.g57019f91.el9.x86_64 35/39 2026-02-21T14:32:28.313 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-base-2:20.3.0-5076.g57019f91.el9.x86_64 35/39 2026-02-21T14:32:28.313 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-21T14:32:28.313 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-21T14:32:28.314 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:32:28.317 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-selinux-2:20.3.0-5076.g57019f91.el9.x86_64 36/39 2026-02-21T14:32:32.844 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5076.g57019f91.el9.x86_64 36/39 2026-02-21T14:32:32.844 INFO:teuthology.orchestra.run.trial130.stdout:skipping the directory /sys 2026-02-21T14:32:32.844 INFO:teuthology.orchestra.run.trial130.stdout:skipping the directory /proc 2026-02-21T14:32:32.844 INFO:teuthology.orchestra.run.trial130.stdout:skipping the directory /mnt 2026-02-21T14:32:32.845 INFO:teuthology.orchestra.run.trial130.stdout:skipping the directory /var/tmp 2026-02-21T14:32:32.845 INFO:teuthology.orchestra.run.trial130.stdout:skipping the directory /home 2026-02-21T14:32:32.845 INFO:teuthology.orchestra.run.trial130.stdout:skipping the directory /root 2026-02-21T14:32:32.845 INFO:teuthology.orchestra.run.trial130.stdout:skipping the directory /tmp 2026-02-21T14:32:32.845 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:32:33.336 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-radosgw-2:20.3.0-5076.g57019f91.el9.x86_64 37/39 2026-02-21T14:32:33.359 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5076.g57019f91.el9.x86_64 37/39 2026-02-21T14:32:33.359 INFO:teuthology.orchestra.run.trial130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T14:32:33.359 INFO:teuthology.orchestra.run.trial130.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-21T14:32:33.359 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-21T14:32:33.360 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-21T14:32:33.360 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:32:33.361 INFO:teuthology.orchestra.run.trial130.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-21T14:32:33.372 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-21T14:32:33.373 INFO:teuthology.orchestra.run.trial130.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-21T14:32:33.425 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-21T14:32:33.425 INFO:teuthology.orchestra.run.trial130.stdout:Creating group 'qat' with GID 992. 2026-02-21T14:32:33.425 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:32:33.721 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-base-2:20.3.0-5076.g57019f91.el9.x86_64 1/39 2026-02-21T14:32:33.721 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-common-2:20.3.0-5076.g57019f91.el9.x86_64 2/39 2026-02-21T14:32:33.721 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-radosgw-2:20.3.0-5076.g57019f91.el9.x86_64 3/39 2026-02-21T14:32:33.721 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-selinux-2:20.3.0-5076.g57019f91.el9.x86_64 4/39 2026-02-21T14:32:33.721 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libcephfs-daemon-2:20.3.0-5076.g57019f91.el9.x86_6 5/39 2026-02-21T14:32:33.721 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5076.g57019f91.el9.x86_6 6/39 2026-02-21T14:32:33.721 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libcephfs2-2:20.3.0-5076.g57019f91.el9.x86_64 7/39 2026-02-21T14:32:33.721 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libradosstriper1-2:20.3.0-5076.g57019f91.el9.x86_6 8/39 2026-02-21T14:32:33.721 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : librgw2-2:20.3.0-5076.g57019f91.el9.x86_64 9/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5076.g57019f91.el9. 10/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-ceph-common-2:20.3.0-5076.g57019f91.el9.x8 11/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-cephfs-2:20.3.0-5076.g57019f91.el9.x86_64 12/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-rados-2:20.3.0-5076.g57019f91.el9.x86_64 13/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-rbd-2:20.3.0-5076.g57019f91.el9.x86_64 14/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-rgw-2:20.3.0-5076.g57019f91.el9.x86_64 15/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-21T14:32:33.722 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-21T14:32:33.723 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-21T14:32:33.724 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-21T14:32:33.724 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : librados2-2:20.3.0-5076.g57019f91.el9.x86_64 36/39 2026-02-21T14:32:33.724 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-21T14:32:33.724 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : librbd1-2:20.3.0-5076.g57019f91.el9.x86_64 38/39 2026-02-21T14:32:33.785 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-21T14:32:33.785 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:32:33.785 INFO:teuthology.orchestra.run.trial130.stdout:Upgraded: 2026-02-21T14:32:33.785 INFO:teuthology.orchestra.run.trial130.stdout: librados2-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout: librbd1-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout:Installed: 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout: ceph-base-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout: ceph-common-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout: ceph-radosgw-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout: ceph-selinux-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout: libcephfs-daemon-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.786 INFO:teuthology.orchestra.run.trial130.stdout: libcephfs-proxy2-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.787 INFO:teuthology.orchestra.run.trial130.stdout: libcephfs2-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.787 INFO:teuthology.orchestra.run.trial130.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-21T14:32:33.787 INFO:teuthology.orchestra.run.trial130.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-21T14:32:33.787 INFO:teuthology.orchestra.run.trial130.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-21T14:32:33.787 INFO:teuthology.orchestra.run.trial130.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-21T14:32:33.787 INFO:teuthology.orchestra.run.trial130.stdout: libradosstriper1-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.787 INFO:teuthology.orchestra.run.trial130.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-21T14:32:33.787 INFO:teuthology.orchestra.run.trial130.stdout: librgw2-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.787 INFO:teuthology.orchestra.run.trial130.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: python3-ceph-argparse-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: python3-ceph-common-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: python3-cephfs-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: python3-rados-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: python3-rbd-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: python3-rgw-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-21T14:32:33.788 INFO:teuthology.orchestra.run.trial130.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-21T14:32:33.789 INFO:teuthology.orchestra.run.trial130.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-21T14:32:33.789 INFO:teuthology.orchestra.run.trial130.stdout: re2-1:20211101-20.el9.x86_64 2026-02-21T14:32:33.789 INFO:teuthology.orchestra.run.trial130.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-21T14:32:33.789 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:32:33.789 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:32:33.828 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-test 2026-02-21T14:32:34.137 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:01:02 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:32:34.222 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:32:34.222 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:32:34.223 INFO:teuthology.orchestra.run.trial130.stdout: Package Arch Version Repository Size 2026-02-21T14:32:34.223 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:32:34.223 INFO:teuthology.orchestra.run.trial130.stdout:Installing: 2026-02-21T14:32:34.223 INFO:teuthology.orchestra.run.trial130.stdout: ceph-test x86_64 2:20.3.0-5076.g57019f91.el9 ceph 126 M 2026-02-21T14:32:34.223 INFO:teuthology.orchestra.run.trial130.stdout:Installing dependencies: 2026-02-21T14:32:34.223 INFO:teuthology.orchestra.run.trial130.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-21T14:32:34.223 INFO:teuthology.orchestra.run.trial130.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-21T14:32:34.223 INFO:teuthology.orchestra.run.trial130.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-21T14:32:34.223 INFO:teuthology.orchestra.run.trial130.stdout: libcephsqlite x86_64 2:20.3.0-5076.g57019f91.el9 ceph 153 k 2026-02-21T14:32:34.223 INFO:teuthology.orchestra.run.trial130.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-21T14:32:34.224 INFO:teuthology.orchestra.run.trial130.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-21T14:32:34.224 INFO:teuthology.orchestra.run.trial130.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-21T14:32:34.224 INFO:teuthology.orchestra.run.trial130.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-21T14:32:34.224 INFO:teuthology.orchestra.run.trial130.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-21T14:32:34.224 INFO:teuthology.orchestra.run.trial130.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-21T14:32:34.224 INFO:teuthology.orchestra.run.trial130.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-21T14:32:34.224 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:32:34.224 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:32:34.224 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:32:34.224 INFO:teuthology.orchestra.run.trial130.stdout:Install 12 Packages 2026-02-21T14:32:34.224 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:32:34.225 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 131 M 2026-02-21T14:32:34.225 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 977 M 2026-02-21T14:32:34.225 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:32:35.113 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] hwloc-libs-2.4.1-6.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/hwloc-libs-2.4.1-6.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:32:35.385 INFO:teuthology.orchestra.run.trial130.stdout:(1/12): libcephsqlite-20.3.0-5076.g57019f91.el9 240 kB/s | 153 kB 00:00 2026-02-21T14:32:35.528 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] hwloc-libs-2.4.1-6.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/hwloc-libs-2.4.1-6.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:32:35.725 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] libatomic-11.5.0-14.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libatomic-11.5.0-14.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:32:36.004 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] hwloc-libs-2.4.1-6.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/hwloc-libs-2.4.1-6.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:32:36.067 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] libatomic-11.5.0-14.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libatomic-11.5.0-14.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:32:36.475 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] libatomic-11.5.0-14.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libatomic-11.5.0-14.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:32:36.652 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] hwloc-libs-2.4.1-6.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/hwloc-libs-2.4.1-6.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:32:36.875 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] libatomic-11.5.0-14.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libatomic-11.5.0-14.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:32:37.122 INFO:teuthology.orchestra.run.trial130.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 13 kB/s | 23 kB 00:01 2026-02-21T14:32:37.395 INFO:teuthology.orchestra.run.trial130.stdout:(3/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 812 kB/s | 2.1 MB 00:02 2026-02-21T14:32:37.465 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] lksctp-tools-1.0.19-2.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/lksctp-tools-1.0.19-2.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:32:37.806 INFO:teuthology.orchestra.run.trial130.stdout:[MIRROR] lksctp-tools-1.0.19-2.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/lksctp-tools-1.0.19-2.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-21T14:32:38.253 INFO:teuthology.orchestra.run.trial130.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 612 kB/s | 525 kB 00:00 2026-02-21T14:32:38.520 INFO:teuthology.orchestra.run.trial130.stdout:(5/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 67 kB/s | 94 kB 00:01 2026-02-21T14:32:38.987 INFO:teuthology.orchestra.run.trial130.stdout:(6/12): libubsan-11.5.0-14.el9.x86_64.rpm 240 kB/s | 176 kB 00:00 2026-02-21T14:32:39.274 INFO:teuthology.orchestra.run.trial130.stdout:(7/12): libxslt-1.1.34-12.el9.x86_64.rpm 309 kB/s | 233 kB 00:00 2026-02-21T14:32:39.920 INFO:teuthology.orchestra.run.trial130.stdout:(8/12): protobuf-3.14.0-17.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2026-02-21T14:32:40.064 INFO:teuthology.orchestra.run.trial130.stdout:(9/12): socat-1.7.4.1-8.el9.x86_64.rpm 384 kB/s | 303 kB 00:00 2026-02-21T14:32:40.332 INFO:teuthology.orchestra.run.trial130.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 454 kB/s | 122 kB 00:00 2026-02-21T14:32:40.540 INFO:teuthology.orchestra.run.trial130.stdout:(11/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 103 kB/s | 64 kB 00:00 2026-02-21T14:39:23.924 INFO:teuthology.orchestra.run.trial130.stdout:(12/12): ceph-test-20.3.0-5076.g57019f91.el9.x8 316 kB/s | 126 MB 06:49 2026-02-21T14:39:23.925 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:39:23.926 INFO:teuthology.orchestra.run.trial130.stdout:Total 327 kB/s | 131 MB 06:49 2026-02-21T14:39:23.999 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:39:24.010 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:39:24.010 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:39:24.441 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:39:24.442 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:39:24.900 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:39:24.909 INFO:teuthology.orchestra.run.trial130.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-21T14:39:24.918 INFO:teuthology.orchestra.run.trial130.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-21T14:39:24.923 INFO:teuthology.orchestra.run.trial130.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-21T14:39:24.927 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-21T14:39:24.929 INFO:teuthology.orchestra.run.trial130.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-21T14:39:24.934 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-21T14:39:24.935 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-21T14:39:24.952 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-21T14:39:24.954 INFO:teuthology.orchestra.run.trial130.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-21T14:39:24.965 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-21T14:39:24.975 INFO:teuthology.orchestra.run.trial130.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-21T14:39:24.976 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libcephsqlite-2:20.3.0-5076.g57019f91.el9.x86_64 11/12 2026-02-21T14:39:32.035 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5076.g57019f91.el9.x86_64 11/12 2026-02-21T14:39:32.053 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-test-2:20.3.0-5076.g57019f91.el9.x86_64 12/12 2026-02-21T14:39:32.814 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-test-2:20.3.0-5076.g57019f91.el9.x86_64 12/12 2026-02-21T14:39:32.814 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-test-2:20.3.0-5076.g57019f91.el9.x86_64 1/12 2026-02-21T14:39:32.814 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libcephsqlite-2:20.3.0-5076.g57019f91.el9.x86_64 2/12 2026-02-21T14:39:32.814 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-21T14:39:32.814 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-21T14:39:32.814 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-21T14:39:32.814 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-21T14:39:32.814 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-21T14:39:32.814 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-21T14:39:32.815 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-21T14:39:32.815 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-21T14:39:32.815 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-21T14:39:32.869 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-21T14:39:32.869 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:39:32.869 INFO:teuthology.orchestra.run.trial130.stdout:Installed: 2026-02-21T14:39:32.870 INFO:teuthology.orchestra.run.trial130.stdout: ceph-test-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:39:32.870 INFO:teuthology.orchestra.run.trial130.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-21T14:39:32.870 INFO:teuthology.orchestra.run.trial130.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-21T14:39:32.870 INFO:teuthology.orchestra.run.trial130.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-21T14:39:32.870 INFO:teuthology.orchestra.run.trial130.stdout: libcephsqlite-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:39:32.870 INFO:teuthology.orchestra.run.trial130.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-21T14:39:32.870 INFO:teuthology.orchestra.run.trial130.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-21T14:39:32.870 INFO:teuthology.orchestra.run.trial130.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-21T14:39:32.870 INFO:teuthology.orchestra.run.trial130.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-21T14:39:32.870 INFO:teuthology.orchestra.run.trial130.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-21T14:39:32.870 INFO:teuthology.orchestra.run.trial130.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-21T14:39:32.871 INFO:teuthology.orchestra.run.trial130.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-21T14:39:32.871 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:39:32.871 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:39:32.912 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph 2026-02-21T14:39:33.217 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:08:01 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:39:33.303 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:39:33.306 INFO:teuthology.orchestra.run.trial130.stdout:======================================================================================= 2026-02-21T14:39:33.306 INFO:teuthology.orchestra.run.trial130.stdout: Package Arch Version Repository Size 2026-02-21T14:39:33.306 INFO:teuthology.orchestra.run.trial130.stdout:======================================================================================= 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout:Installing: 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout: ceph x86_64 2:20.3.0-5076.g57019f91.el9 ceph 6.5 k 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout:Installing dependencies: 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5076.g57019f91.el9 ceph-noarch 57 k 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mds x86_64 2:20.3.0-5076.g57019f91.el9 ceph 2.0 M 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr x86_64 2:20.3.0-5076.g57019f91.el9 ceph 937 k 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5076.g57019f91.el9 ceph-noarch 293 k 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mon x86_64 2:20.3.0-5076.g57019f91.el9 ceph 4.8 M 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout: ceph-osd x86_64 2:20.3.0-5076.g57019f91.el9 ceph 163 k 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout: ceph-osd-classic x86_64 2:20.3.0-5076.g57019f91.el9 ceph 15 M 2026-02-21T14:39:33.307 INFO:teuthology.orchestra.run.trial130.stdout: ceph-osd-crimson x86_64 2:20.3.0-5076.g57019f91.el9 ceph 87 M 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5076.g57019f91.el9 ceph-noarch 17 k 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: cephadm noarch 2:20.3.0-5076.g57019f91.el9 ceph-noarch 1.0 M 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-21T14:39:33.308 INFO:teuthology.orchestra.run.trial130.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-21T14:39:33.309 INFO:teuthology.orchestra.run.trial130.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-21T14:39:33.310 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-21T14:39:33.311 INFO:teuthology.orchestra.run.trial130.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-21T14:39:33.312 INFO:teuthology.orchestra.run.trial130.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-21T14:39:33.313 INFO:teuthology.orchestra.run.trial130.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout:Installing weak dependencies: 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5076.g57019f91.el9 ceph-noarch 183 k 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5076.g57019f91.el9 ceph-noarch 15 M 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5076.g57019f91.el9 ceph-noarch 7.4 M 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5076.g57019f91.el9 ceph-noarch 22 k 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-rook noarch 2:20.3.0-5076.g57019f91.el9 ceph-noarch 50 k 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: ceph-volume noarch 2:20.3.0-5076.g57019f91.el9 ceph-noarch 299 k 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-21T14:39:33.314 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:39:33.315 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:39:33.315 INFO:teuthology.orchestra.run.trial130.stdout:======================================================================================= 2026-02-21T14:39:33.315 INFO:teuthology.orchestra.run.trial130.stdout:Install 96 Packages 2026-02-21T14:39:33.315 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:39:33.316 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 189 M 2026-02-21T14:39:33.316 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 1.4 G 2026-02-21T14:39:33.316 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:39:33.951 INFO:teuthology.orchestra.run.trial130.stdout:(1/96): ceph-20.3.0-5076.g57019f91.el9.x86_64.r 23 kB/s | 6.5 kB 00:00 2026-02-21T14:39:34.921 INFO:teuthology.orchestra.run.trial130.stdout:(2/96): ceph-mgr-20.3.0-5076.g57019f91.el9.x86_ 750 kB/s | 937 kB 00:01 2026-02-21T14:39:35.060 INFO:teuthology.orchestra.run.trial130.stdout:(3/96): ceph-osd-20.3.0-5076.g57019f91.el9.x86_ 1.1 MB/s | 163 kB 00:00 2026-02-21T14:39:36.988 INFO:teuthology.orchestra.run.trial130.stdout:(4/96): ceph-mds-20.3.0-5076.g57019f91.el9.x86_ 624 kB/s | 2.0 MB 00:03 2026-02-21T14:39:43.786 INFO:teuthology.orchestra.run.trial130.stdout:(5/96): ceph-mon-20.3.0-5076.g57019f91.el9.x86_ 504 kB/s | 4.8 MB 00:09 2026-02-21T14:39:43.927 INFO:teuthology.orchestra.run.trial130.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5076.g57 407 kB/s | 57 kB 00:00 2026-02-21T14:39:44.208 INFO:teuthology.orchestra.run.trial130.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5076.g57019f91. 652 kB/s | 183 kB 00:00 2026-02-21T14:40:04.787 INFO:teuthology.orchestra.run.trial130.stdout:(8/96): ceph-osd-classic-20.3.0-5076.g57019f91. 508 kB/s | 15 MB 00:29 2026-02-21T14:40:18.024 INFO:teuthology.orchestra.run.trial130.stdout:(9/96): ceph-mgr-dashboard-20.3.0-5076.g57019f9 452 kB/s | 15 MB 00:33 2026-02-21T14:40:18.093 INFO:teuthology.orchestra.run.trial130.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5076.g57019f 322 kB/s | 22 kB 00:00 2026-02-21T14:40:19.061 INFO:teuthology.orchestra.run.trial130.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5076.g570 302 kB/s | 293 kB 00:00 2026-02-21T14:40:19.271 INFO:teuthology.orchestra.run.trial130.stdout:(12/96): ceph-mgr-rook-20.3.0-5076.g57019f91.el 239 kB/s | 50 kB 00:00 2026-02-21T14:40:19.342 INFO:teuthology.orchestra.run.trial130.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5076.g57 243 kB/s | 17 kB 00:00 2026-02-21T14:40:20.173 INFO:teuthology.orchestra.run.trial130.stdout:(14/96): ceph-volume-20.3.0-5076.g57019f91.el9. 359 kB/s | 299 kB 00:00 2026-02-21T14:40:22.022 INFO:teuthology.orchestra.run.trial130.stdout:(15/96): cephadm-20.3.0-5076.g57019f91.el9.noar 562 kB/s | 1.0 MB 00:01 2026-02-21T14:40:22.131 INFO:teuthology.orchestra.run.trial130.stdout:(16/96): ceph-mgr-diskprediction-local-20.3.0-5 438 kB/s | 7.4 MB 00:17 2026-02-21T14:40:22.590 INFO:teuthology.orchestra.run.trial130.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 88 kB/s | 40 kB 00:00 2026-02-21T14:40:22.698 INFO:teuthology.orchestra.run.trial130.stdout:(18/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 519 kB/s | 351 kB 00:00 2026-02-21T14:40:22.767 INFO:teuthology.orchestra.run.trial130.stdout:(19/96): libconfig-1.7.2-9.el9.x86_64.rpm 407 kB/s | 72 kB 00:00 2026-02-21T14:40:22.879 INFO:teuthology.orchestra.run.trial130.stdout:(20/96): libgfortran-11.5.0-14.el9.x86_64.rpm 4.3 MB/s | 794 kB 00:00 2026-02-21T14:40:22.945 INFO:teuthology.orchestra.run.trial130.stdout:(21/96): libquadmath-11.5.0-14.el9.x86_64.rpm 1.0 MB/s | 184 kB 00:00 2026-02-21T14:40:22.979 INFO:teuthology.orchestra.run.trial130.stdout:(22/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2026-02-21T14:40:23.080 INFO:teuthology.orchestra.run.trial130.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2026-02-21T14:40:23.180 INFO:teuthology.orchestra.run.trial130.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 1.3 MB/s | 135 kB 00:00 2026-02-21T14:40:23.204 INFO:teuthology.orchestra.run.trial130.stdout:(25/96): python3-cryptography-36.0.1-5.el9.x86_ 4.8 MB/s | 1.2 MB 00:00 2026-02-21T14:40:23.281 INFO:teuthology.orchestra.run.trial130.stdout:(26/96): python3-pyparsing-2.4.7-9.el9.noarch.r 1.5 MB/s | 150 kB 00:00 2026-02-21T14:40:23.304 INFO:teuthology.orchestra.run.trial130.stdout:(27/96): python3-requests-2.25.1-10.el9.noarch. 1.2 MB/s | 126 kB 00:00 2026-02-21T14:40:23.382 INFO:teuthology.orchestra.run.trial130.stdout:(28/96): python3-urllib3-1.26.5-7.el9.noarch.rp 2.1 MB/s | 218 kB 00:00 2026-02-21T14:40:23.404 INFO:teuthology.orchestra.run.trial130.stdout:(29/96): unzip-6.0-59.el9.x86_64.rpm 1.8 MB/s | 182 kB 00:00 2026-02-21T14:40:23.435 INFO:teuthology.orchestra.run.trial130.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 960 kB/s | 30 kB 00:00 2026-02-21T14:40:23.482 INFO:teuthology.orchestra.run.trial130.stdout:(31/96): zip-3.0-35.el9.x86_64.rpm 2.6 MB/s | 266 kB 00:00 2026-02-21T14:40:23.508 INFO:teuthology.orchestra.run.trial130.stdout:(32/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 41 MB/s | 3.0 MB 00:00 2026-02-21T14:40:23.509 INFO:teuthology.orchestra.run.trial130.stdout:(33/96): flexiblas-openblas-openmp-3.0.4-9.el9. 548 kB/s | 15 kB 00:00 2026-02-21T14:40:23.521 INFO:teuthology.orchestra.run.trial130.stdout:(34/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 20 MB/s | 246 kB 00:00 2026-02-21T14:40:23.531 INFO:teuthology.orchestra.run.trial130.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 4.1 MB/s | 42 kB 00:00 2026-02-21T14:40:23.548 INFO:teuthology.orchestra.run.trial130.stdout:(36/96): lua-5.4.4-4.el9.x86_64.rpm 4.8 MB/s | 188 kB 00:00 2026-02-21T14:40:23.560 INFO:teuthology.orchestra.run.trial130.stdout:(37/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 181 MB/s | 5.3 MB 00:00 2026-02-21T14:40:23.573 INFO:teuthology.orchestra.run.trial130.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 18 MB/s | 244 kB 00:00 2026-02-21T14:40:23.588 INFO:teuthology.orchestra.run.trial130.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 16 MB/s | 249 kB 00:00 2026-02-21T14:40:23.600 INFO:teuthology.orchestra.run.trial130.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 4.2 MB/s | 48 kB 00:00 2026-02-21T14:40:23.615 INFO:teuthology.orchestra.run.trial130.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 12 MB/s | 177 kB 00:00 2026-02-21T14:40:23.630 INFO:teuthology.orchestra.run.trial130.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 79 MB/s | 1.2 MB 00:00 2026-02-21T14:40:23.642 INFO:teuthology.orchestra.run.trial130.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 2.8 MB/s | 35 kB 00:00 2026-02-21T14:40:23.693 INFO:teuthology.orchestra.run.trial130.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 121 MB/s | 6.1 MB 00:00 2026-02-21T14:40:23.700 INFO:teuthology.orchestra.run.trial130.stdout:(45/96): python3-babel-2.9.1-2.el9.noarch.rpm 39 MB/s | 6.0 MB 00:00 2026-02-21T14:40:23.707 INFO:teuthology.orchestra.run.trial130.stdout:(46/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 32 MB/s | 442 kB 00:00 2026-02-21T14:40:23.712 INFO:teuthology.orchestra.run.trial130.stdout:(47/96): python3-packaging-20.9-5.el9.noarch.rp 6.6 MB/s | 77 kB 00:00 2026-02-21T14:40:23.719 INFO:teuthology.orchestra.run.trial130.stdout:(48/96): python3-protobuf-3.14.0-17.el9.noarch. 21 MB/s | 267 kB 00:00 2026-02-21T14:40:23.724 INFO:teuthology.orchestra.run.trial130.stdout:(49/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 13 MB/s | 159 kB 00:00 2026-02-21T14:40:23.732 INFO:teuthology.orchestra.run.trial130.stdout:(50/96): python3-pyasn1-modules-0.4.8-6.el9.noa 21 MB/s | 279 kB 00:00 2026-02-21T14:40:23.735 INFO:teuthology.orchestra.run.trial130.stdout:(51/96): python3-requests-oauthlib-1.3.0-12.el9 4.8 MB/s | 54 kB 00:00 2026-02-21T14:40:23.746 INFO:teuthology.orchestra.run.trial130.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 3.7 MB/s | 42 kB 00:00 2026-02-21T14:40:23.758 INFO:teuthology.orchestra.run.trial130.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 16 MB/s | 189 kB 00:00 2026-02-21T14:40:23.769 INFO:teuthology.orchestra.run.trial130.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 8.0 MB/s | 90 kB 00:00 2026-02-21T14:40:23.830 INFO:teuthology.orchestra.run.trial130.stdout:(55/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 198 MB/s | 19 MB 00:00 2026-02-21T14:40:24.011 INFO:teuthology.orchestra.run.trial130.stdout:(56/96): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2026-02-21T14:40:24.465 INFO:teuthology.orchestra.run.trial130.stdout:(57/96): protobuf-compiler-3.14.0-17.el9.x86_64 1.3 MB/s | 862 kB 00:00 2026-02-21T14:40:24.520 INFO:teuthology.orchestra.run.trial130.stdout:(58/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.1 MB/s | 551 kB 00:00 2026-02-21T14:40:24.673 INFO:teuthology.orchestra.run.trial130.stdout:(59/96): grpc-data-1.46.7-10.el9.noarch.rpm 93 kB/s | 19 kB 00:00 2026-02-21T14:40:24.878 INFO:teuthology.orchestra.run.trial130.stdout:(60/96): luarocks-3.9.2-5.el9.noarch.rpm 423 kB/s | 151 kB 00:00 2026-02-21T14:40:25.087 INFO:teuthology.orchestra.run.trial130.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 141 kB/s | 29 kB 00:00 2026-02-21T14:40:25.176 INFO:teuthology.orchestra.run.trial130.stdout:(62/96): python3-asyncssh-2.13.2-5.el9.noarch.r 1.1 MB/s | 548 kB 00:00 2026-02-21T14:40:25.365 INFO:teuthology.orchestra.run.trial130.stdout:(63/96): python3-backports-tarfile-1.2.0-1.el9. 217 kB/s | 60 kB 00:00 2026-02-21T14:40:25.392 INFO:teuthology.orchestra.run.trial130.stdout:(64/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 202 kB/s | 43 kB 00:00 2026-02-21T14:40:25.537 INFO:teuthology.orchestra.run.trial130.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 97 kB/s | 14 kB 00:00 2026-02-21T14:40:25.574 INFO:teuthology.orchestra.run.trial130.stdout:(66/96): python3-cachetools-4.2.4-1.el9.noarch. 154 kB/s | 32 kB 00:00 2026-02-21T14:40:25.899 INFO:teuthology.orchestra.run.trial130.stdout:(67/96): python3-cheroot-10.0.1-4.el9.noarch.rp 479 kB/s | 173 kB 00:00 2026-02-21T14:40:26.005 INFO:teuthology.orchestra.run.trial130.stdout:(68/96): python3-cherrypy-18.6.1-2.el9.noarch.r 831 kB/s | 358 kB 00:00 2026-02-21T14:40:26.382 INFO:teuthology.orchestra.run.trial130.stdout:(69/96): python3-google-auth-2.45.0-1.el9.noarc 527 kB/s | 254 kB 00:00 2026-02-21T14:40:26.733 INFO:teuthology.orchestra.run.trial130.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 411 kB/s | 144 kB 00:00 2026-02-21T14:40:26.747 INFO:teuthology.orchestra.run.trial130.stdout:(71/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.7 MB/s | 2.0 MB 00:00 2026-02-21T14:40:27.073 INFO:teuthology.orchestra.run.trial130.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 172 kB/s | 56 kB 00:00 2026-02-21T14:40:27.090 INFO:teuthology.orchestra.run.trial130.stdout:(73/96): python3-influxdb-5.3.1-1.el9.noarch.rp 389 kB/s | 139 kB 00:00 2026-02-21T14:40:27.287 INFO:teuthology.orchestra.run.trial130.stdout:(74/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 50 kB/s | 11 kB 00:00 2026-02-21T14:40:27.393 INFO:teuthology.orchestra.run.trial130.stdout:(75/96): python3-jaraco-classes-3.2.1-5.el9.noa 58 kB/s | 18 kB 00:00 2026-02-21T14:40:27.497 INFO:teuthology.orchestra.run.trial130.stdout:(76/96): python3-jaraco-collections-3.0.0-8.el9 110 kB/s | 23 kB 00:00 2026-02-21T14:40:27.603 INFO:teuthology.orchestra.run.trial130.stdout:(77/96): python3-jaraco-context-6.0.1-3.el9.noa 93 kB/s | 20 kB 00:00 2026-02-21T14:40:27.819 INFO:teuthology.orchestra.run.trial130.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 123 kB/s | 26 kB 00:00 2026-02-21T14:40:27.828 INFO:teuthology.orchestra.run.trial130.stdout:(79/96): python3-jaraco-functools-3.5.0-2.el9.n 59 kB/s | 19 kB 00:00 2026-02-21T14:40:28.117 INFO:teuthology.orchestra.run.trial130.stdout:(80/96): python3-more-itertools-8.12.0-2.el9.no 272 kB/s | 79 kB 00:00 2026-02-21T14:40:28.397 INFO:teuthology.orchestra.run.trial130.stdout:(81/96): python3-kubernetes-26.1.0-3.el9.noarch 1.8 MB/s | 1.0 MB 00:00 2026-02-21T14:40:28.439 INFO:teuthology.orchestra.run.trial130.stdout:(82/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 268 kB/s | 86 kB 00:00 2026-02-21T14:40:28.671 INFO:teuthology.orchestra.run.trial130.stdout:(83/96): python3-portend-3.1.0-2.el9.noarch.rpm 71 kB/s | 16 kB 00:00 2026-02-21T14:40:28.747 INFO:teuthology.orchestra.run.trial130.stdout:(84/96): python3-natsort-7.1.1-5.el9.noarch.rpm 165 kB/s | 58 kB 00:00 2026-02-21T14:40:28.957 INFO:teuthology.orchestra.run.trial130.stdout:(85/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 314 kB/s | 90 kB 00:00 2026-02-21T14:40:28.960 INFO:teuthology.orchestra.run.trial130.stdout:(86/96): python3-repoze-lru-0.7-16.el9.noarch.r 144 kB/s | 31 kB 00:00 2026-02-21T14:40:29.238 INFO:teuthology.orchestra.run.trial130.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 213 kB/s | 59 kB 00:00 2026-02-21T14:40:29.319 INFO:teuthology.orchestra.run.trial130.stdout:(88/96): python3-routes-2.5.1-5.el9.noarch.rpm 525 kB/s | 188 kB 00:00 2026-02-21T14:40:29.534 INFO:teuthology.orchestra.run.trial130.stdout:(89/96): python3-tempora-5.0.0-2.el9.noarch.rpm 166 kB/s | 36 kB 00:00 2026-02-21T14:40:29.600 INFO:teuthology.orchestra.run.trial130.stdout:(90/96): python3-saml-1.16.0-1.el9.noarch.rpm 345 kB/s | 125 kB 00:00 2026-02-21T14:40:29.913 INFO:teuthology.orchestra.run.trial130.stdout:(91/96): python3-typing-extensions-4.15.0-1.el9 228 kB/s | 86 kB 00:00 2026-02-21T14:40:29.917 INFO:teuthology.orchestra.run.trial130.stdout:(92/96): python3-websocket-client-1.2.3-2.el9.n 282 kB/s | 90 kB 00:00 2026-02-21T14:40:30.194 INFO:teuthology.orchestra.run.trial130.stdout:(93/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 172 kB/s | 48 kB 00:00 2026-02-21T14:40:30.236 INFO:teuthology.orchestra.run.trial130.stdout:(94/96): python3-xmltodict-0.12.0-15.el9.noarch 69 kB/s | 22 kB 00:00 2026-02-21T14:40:30.403 INFO:teuthology.orchestra.run.trial130.stdout:(95/96): python3-zc-lockfile-2.0-10.el9.noarch. 96 kB/s | 20 kB 00:00 2026-02-21T14:43:39.702 INFO:teuthology.orchestra.run.trial130.stdout:(96/96): ceph-osd-crimson-20.3.0-5076.g57019f91 366 kB/s | 87 MB 04:02 2026-02-21T14:43:39.703 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:43:39.704 INFO:teuthology.orchestra.run.trial130.stdout:Total 785 kB/s | 189 MB 04:06 2026-02-21T14:43:40.076 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:43:40.115 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:43:40.116 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:43:40.812 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:43:40.813 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:43:41.546 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:43:41.548 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-21T14:43:41.552 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-21T14:43:41.573 INFO:teuthology.orchestra.run.trial130.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-21T14:43:41.607 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-21T14:43:41.613 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-21T14:43:41.616 INFO:teuthology.orchestra.run.trial130.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-21T14:43:41.622 INFO:teuthology.orchestra.run.trial130.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-21T14:43:41.645 INFO:teuthology.orchestra.run.trial130.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-21T14:43:41.657 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-21T14:43:41.661 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-21T14:43:41.667 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-21T14:43:41.669 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-21T14:43:41.680 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-21T14:43:41.687 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-21T14:43:41.711 INFO:teuthology.orchestra.run.trial130.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-21T14:43:41.714 INFO:teuthology.orchestra.run.trial130.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-21T14:43:41.764 INFO:teuthology.orchestra.run.trial130.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-21T14:43:41.777 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-21T14:43:41.779 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-21T14:43:41.783 INFO:teuthology.orchestra.run.trial130.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-21T14:43:41.789 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-21T14:43:41.793 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-21T14:43:41.796 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-21T14:43:41.811 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-21T14:43:41.815 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-21T14:43:41.820 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-21T14:43:41.832 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-21T14:43:41.840 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-21T14:43:41.847 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-21T14:43:41.866 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-21T14:43:41.877 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-21T14:43:41.893 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-21T14:43:41.897 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-21T14:43:41.905 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-21T14:43:41.945 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-21T14:43:42.268 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-21T14:43:42.278 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-21T14:43:42.282 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-21T14:43:42.288 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-21T14:43:42.289 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-21T14:43:42.317 INFO:teuthology.orchestra.run.trial130.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-21T14:43:42.360 INFO:teuthology.orchestra.run.trial130.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-21T14:43:42.369 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-21T14:43:42.374 INFO:teuthology.orchestra.run.trial130.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-21T14:43:42.378 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-21T14:43:42.384 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-21T14:43:42.387 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-21T14:43:42.394 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-21T14:43:42.397 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-21T14:43:42.425 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-21T14:43:42.435 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-21T14:43:42.442 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-21T14:43:42.449 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-21T14:43:42.483 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-21T14:43:42.706 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-21T14:43:42.730 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-21T14:43:42.734 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-21T14:43:42.793 INFO:teuthology.orchestra.run.trial130.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-21T14:43:42.795 INFO:teuthology.orchestra.run.trial130.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-21T14:43:42.816 INFO:teuthology.orchestra.run.trial130.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-21T14:43:43.140 INFO:teuthology.orchestra.run.trial130.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-21T14:43:43.213 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-21T14:43:43.880 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-21T14:43:43.897 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-21T14:43:43.906 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-21T14:43:43.913 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-21T14:43:43.926 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-21T14:43:43.942 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-21T14:43:44.017 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-21T14:43:44.026 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-21T14:43:44.049 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-21T14:43:44.079 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-21T14:43:44.129 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-21T14:43:44.135 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-21T14:43:44.138 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-21T14:43:44.139 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-21T14:43:44.158 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-21T14:43:44.158 INFO:teuthology.orchestra.run.trial130.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-21T14:43:44.158 INFO:teuthology.orchestra.run.trial130.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-21T14:43:44.158 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:44.165 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-21T14:43:44.191 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-21T14:43:44.191 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-21T14:43:44.191 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:44.224 INFO:teuthology.orchestra.run.trial130.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-21T14:43:44.851 INFO:teuthology.orchestra.run.trial130.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-21T14:43:50.305 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-osd-classic-2:20.3.0-5076.g57019f91.el9.x86_6 80/96 2026-02-21T14:43:50.369 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-osd-crimson-2:20.3.0-5076.g57019f91.el9.x86_6 81/96 2026-02-21T14:43:50.371 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-volume-2:20.3.0-5076.g57019f91.el9.noarch 82/96 2026-02-21T14:43:50.384 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-volume-2:20.3.0-5076.g57019f91.el9.noarch 82/96 2026-02-21T14:43:50.384 INFO:teuthology.orchestra.run.trial130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T14:43:50.384 INFO:teuthology.orchestra.run.trial130.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-21T14:43:50.385 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:50.397 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-osd-2:20.3.0-5076.g57019f91.el9.x86_64 83/96 2026-02-21T14:43:50.420 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-osd-2:20.3.0-5076.g57019f91.el9.x86_64 83/96 2026-02-21T14:43:50.420 INFO:teuthology.orchestra.run.trial130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T14:43:50.420 INFO:teuthology.orchestra.run.trial130.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-21T14:43:50.420 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-21T14:43:50.421 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-21T14:43:50.421 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:50.475 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: cephadm-2:20.3.0-5076.g57019f91.el9.noarch 84/96 2026-02-21T14:43:50.477 INFO:teuthology.orchestra.run.trial130.stdout: Installing : cephadm-2:20.3.0-5076.g57019f91.el9.noarch 84/96 2026-02-21T14:43:50.487 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5076.g57019f91.el9 85/96 2026-02-21T14:43:50.510 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5076.g57019f91.el 86/96 2026-02-21T14:43:50.511 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5076.g57019f91.el9.noarc 87/96 2026-02-21T14:43:52.083 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5076.g57019f91.el9.noarc 87/96 2026-02-21T14:43:52.090 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5076.g57019f91.el9.noa 88/96 2026-02-21T14:43:52.552 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5076.g57019f91.el9.noa 88/96 2026-02-21T14:43:52.553 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5076.g57019 89/96 2026-02-21T14:43:52.564 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5076.g57019 89/96 2026-02-21T14:43:52.564 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5076.g57019f91.el9.noa 90/96 2026-02-21T14:43:52.621 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5076.g57019f91.el9.noa 90/96 2026-02-21T14:43:52.663 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5076.g57019f91.el9. 91/96 2026-02-21T14:43:52.664 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-mgr-2:20.3.0-5076.g57019f91.el9.x86_64 92/96 2026-02-21T14:43:52.684 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5076.g57019f91.el9.x86_64 92/96 2026-02-21T14:43:52.685 INFO:teuthology.orchestra.run.trial130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T14:43:52.685 INFO:teuthology.orchestra.run.trial130.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-21T14:43:52.685 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-21T14:43:52.685 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-21T14:43:52.685 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:52.695 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-mgr-rook-2:20.3.0-5076.g57019f91.el9.noarch 93/96 2026-02-21T14:43:52.923 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5076.g57019f91.el9.noarch 93/96 2026-02-21T14:43:52.925 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-mon-2:20.3.0-5076.g57019f91.el9.x86_64 94/96 2026-02-21T14:43:52.946 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-mon-2:20.3.0-5076.g57019f91.el9.x86_64 94/96 2026-02-21T14:43:52.946 INFO:teuthology.orchestra.run.trial130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T14:43:52.946 INFO:teuthology.orchestra.run.trial130.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-21T14:43:52.946 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-21T14:43:52.946 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-21T14:43:52.946 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:53.034 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-mds-2:20.3.0-5076.g57019f91.el9.x86_64 95/96 2026-02-21T14:43:53.054 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-mds-2:20.3.0-5076.g57019f91.el9.x86_64 95/96 2026-02-21T14:43:53.054 INFO:teuthology.orchestra.run.trial130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T14:43:53.054 INFO:teuthology.orchestra.run.trial130.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-21T14:43:53.054 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-21T14:43:53.054 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-21T14:43:53.055 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:53.055 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-2:20.3.0-5076.g57019f91.el9.x86_64 96/96 2026-02-21T14:43:53.061 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5076.g57019f91.el9.x86_6 96/96 2026-02-21T14:43:53.084 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5076.g57019f91.el9.x86_6 96/96 2026-02-21T14:43:54.361 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-2:20.3.0-5076.g57019f91.el9.x86_64 96/96 2026-02-21T14:43:54.361 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-2:20.3.0-5076.g57019f91.el9.x86_64 1/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-mds-2:20.3.0-5076.g57019f91.el9.x86_64 2/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-mgr-2:20.3.0-5076.g57019f91.el9.x86_64 3/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-mon-2:20.3.0-5076.g57019f91.el9.x86_64 4/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-osd-2:20.3.0-5076.g57019f91.el9.x86_64 5/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-osd-classic-2:20.3.0-5076.g57019f91.el9.x86_6 6/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5076.g57019f91.el9.x86_6 7/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5076.g57019f91.el 8/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5076.g57019f91.el9.noarc 9/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5076.g57019f91.el9.noa 10/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5076.g57019 11/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5076.g57019f91.el9.noa 12/96 2026-02-21T14:43:54.362 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5076.g57019f91.el9. 13/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5076.g57019f91.el9.noarch 14/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5076.g57019f91.el9 15/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-volume-2:20.3.0-5076.g57019f91.el9.noarch 16/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : cephadm-2:20.3.0-5076.g57019f91.el9.noarch 17/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-21T14:43:54.363 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-21T14:43:54.364 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-21T14:43:54.365 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-21T14:43:54.366 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-21T14:43:54.367 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-21T14:43:54.368 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-21T14:43:54.369 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-21T14:43:54.369 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-21T14:43:54.369 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-21T14:43:54.369 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-21T14:43:54.369 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-21T14:43:54.369 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-21T14:43:54.425 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-21T14:43:54.425 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:54.425 INFO:teuthology.orchestra.run.trial130.stdout:Installed: 2026-02-21T14:43:54.426 INFO:teuthology.orchestra.run.trial130.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-21T14:43:54.426 INFO:teuthology.orchestra.run.trial130.stdout: ceph-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:43:54.426 INFO:teuthology.orchestra.run.trial130.stdout: ceph-grafana-dashboards-2:20.3.0-5076.g57019f91.el9.noarch 2026-02-21T14:43:54.426 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mds-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:43:54.426 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:43:54.426 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-cephadm-2:20.3.0-5076.g57019f91.el9.noarch 2026-02-21T14:43:54.426 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-dashboard-2:20.3.0-5076.g57019f91.el9.noarch 2026-02-21T14:43:54.426 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5076.g57019f91.el9.noarch 2026-02-21T14:43:54.426 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-k8sevents-2:20.3.0-5076.g57019f91.el9.noarch 2026-02-21T14:43:54.426 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-modules-core-2:20.3.0-5076.g57019f91.el9.noarch 2026-02-21T14:43:54.426 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mgr-rook-2:20.3.0-5076.g57019f91.el9.noarch 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: ceph-mon-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: ceph-osd-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: ceph-osd-classic-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: ceph-osd-crimson-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: ceph-prometheus-alerts-2:20.3.0-5076.g57019f91.el9.noarch 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: ceph-volume-2:20.3.0-5076.g57019f91.el9.noarch 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: cephadm-2:20.3.0-5076.g57019f91.el9.noarch 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-21T14:43:54.427 INFO:teuthology.orchestra.run.trial130.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: lua-5.4.4-4.el9.x86_64 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-21T14:43:54.428 INFO:teuthology.orchestra.run.trial130.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-21T14:43:54.429 INFO:teuthology.orchestra.run.trial130.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-21T14:43:54.430 INFO:teuthology.orchestra.run.trial130.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-ply-3.11-14.el9.noarch 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-21T14:43:54.431 INFO:teuthology.orchestra.run.trial130.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-21T14:43:54.432 INFO:teuthology.orchestra.run.trial130.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-21T14:43:54.433 INFO:teuthology.orchestra.run.trial130.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-21T14:43:54.433 INFO:teuthology.orchestra.run.trial130.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-21T14:43:54.433 INFO:teuthology.orchestra.run.trial130.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-21T14:43:54.433 INFO:teuthology.orchestra.run.trial130.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-21T14:43:54.433 INFO:teuthology.orchestra.run.trial130.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-21T14:43:54.433 INFO:teuthology.orchestra.run.trial130.stdout: unzip-6.0-59.el9.x86_64 2026-02-21T14:43:54.433 INFO:teuthology.orchestra.run.trial130.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-21T14:43:54.433 INFO:teuthology.orchestra.run.trial130.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-21T14:43:54.433 INFO:teuthology.orchestra.run.trial130.stdout: zip-3.0-35.el9.x86_64 2026-02-21T14:43:54.433 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:54.433 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:43:54.490 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-base 2026-02-21T14:43:54.812 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:22 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:43:54.879 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-base-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:43:54.898 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:43:54.898 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:43:54.898 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:43:54.920 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install cephadm 2026-02-21T14:43:55.230 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:23 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:43:55.296 INFO:teuthology.orchestra.run.trial130.stdout:Package cephadm-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:43:55.314 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:43:55.315 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:43:55.315 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:43:55.337 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-immutable-object-cache 2026-02-21T14:43:55.647 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:23 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:43:55.730 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:43:55.731 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:43:55.731 INFO:teuthology.orchestra.run.trial130.stdout: Package Arch Version Repo Size 2026-02-21T14:43:55.731 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:43:55.731 INFO:teuthology.orchestra.run.trial130.stdout:Installing: 2026-02-21T14:43:55.731 INFO:teuthology.orchestra.run.trial130.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5076.g57019f91.el9 ceph 149 k 2026-02-21T14:43:55.731 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:55.731 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:43:55.732 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:43:55.732 INFO:teuthology.orchestra.run.trial130.stdout:Install 1 Package 2026-02-21T14:43:55.732 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:55.732 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 149 k 2026-02-21T14:43:55.732 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 455 k 2026-02-21T14:43:55.732 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:43:56.224 INFO:teuthology.orchestra.run.trial130.stdout:ceph-immutable-object-cache-20.3.0-5076.g57019f 302 kB/s | 149 kB 00:00 2026-02-21T14:43:56.225 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:43:56.225 INFO:teuthology.orchestra.run.trial130.stdout:Total 302 kB/s | 149 kB 00:00 2026-02-21T14:43:56.225 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:43:56.230 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:43:56.230 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:43:56.261 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:43:56.261 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:43:56.318 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:43:56.319 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5076.g57019f91. 1/1 2026-02-21T14:43:56.343 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5076.g57019f91. 1/1 2026-02-21T14:43:56.343 INFO:teuthology.orchestra.run.trial130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T14:43:56.343 INFO:teuthology.orchestra.run.trial130.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-21T14:43:56.343 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:56.658 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5076.g57019f91. 1/1 2026-02-21T14:43:56.659 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:56.659 INFO:teuthology.orchestra.run.trial130.stdout:Installed: 2026-02-21T14:43:56.659 INFO:teuthology.orchestra.run.trial130.stdout: ceph-immutable-object-cache-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:43:56.659 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:43:56.659 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:43:56.687 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-mgr 2026-02-21T14:43:57.002 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:25 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:43:57.068 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-mgr-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:43:57.087 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:43:57.087 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:43:57.087 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:43:57.109 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-mgr-dashboard 2026-02-21T14:43:57.414 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:25 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:43:57.479 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-mgr-dashboard-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:43:57.498 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:43:57.498 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:43:57.498 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:43:57.520 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-21T14:43:57.830 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:25 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:43:57.895 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:43:57.914 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:43:57.914 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:43:57.914 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:43:57.936 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-mgr-rook 2026-02-21T14:43:58.246 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:26 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:43:58.310 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-mgr-rook-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:43:58.329 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:43:58.329 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:43:58.329 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:43:58.351 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-mgr-cephadm 2026-02-21T14:43:58.659 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:26 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:43:58.724 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-mgr-cephadm-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:43:58.742 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:43:58.743 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:43:58.743 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:43:58.764 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-osd 2026-02-21T14:43:59.075 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:27 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:43:59.140 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-osd-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:43:59.158 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:43:59.159 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:43:59.159 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:43:59.180 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-osd-classic 2026-02-21T14:43:59.495 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:27 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:43:59.560 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-osd-classic-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:43:59.579 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:43:59.579 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:43:59.579 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:43:59.602 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-fuse 2026-02-21T14:43:59.917 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:27 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:00.000 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:00.001 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:00.001 INFO:teuthology.orchestra.run.trial130.stdout: Package Arch Version Repo Size 2026-02-21T14:44:00.001 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:00.001 INFO:teuthology.orchestra.run.trial130.stdout:Installing: 2026-02-21T14:44:00.001 INFO:teuthology.orchestra.run.trial130.stdout: ceph-fuse x86_64 2:20.3.0-5076.g57019f91.el9 ceph 914 k 2026-02-21T14:44:00.001 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:00.001 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:44:00.001 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:00.002 INFO:teuthology.orchestra.run.trial130.stdout:Install 1 Package 2026-02-21T14:44:00.002 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:00.002 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 914 k 2026-02-21T14:44:00.002 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 2.9 M 2026-02-21T14:44:00.002 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:44:01.884 INFO:teuthology.orchestra.run.trial130.stdout:ceph-fuse-20.3.0-5076.g57019f91.el9.x86_64.rpm 486 kB/s | 914 kB 00:01 2026-02-21T14:44:01.884 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:44:01.885 INFO:teuthology.orchestra.run.trial130.stdout:Total 485 kB/s | 914 kB 00:01 2026-02-21T14:44:01.885 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:44:01.889 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:44:01.890 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:44:01.924 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:44:01.924 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:44:02.018 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:44:02.035 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-fuse-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:02.268 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:02.327 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-fuse-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:02.327 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:02.327 INFO:teuthology.orchestra.run.trial130.stdout:Installed: 2026-02-21T14:44:02.328 INFO:teuthology.orchestra.run.trial130.stdout: ceph-fuse-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:44:02.328 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:02.328 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:02.359 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-volume 2026-02-21T14:44:02.671 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:30 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:02.735 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-volume-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:44:02.753 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:02.754 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:44:02.754 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:02.775 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install librados-devel 2026-02-21T14:44:03.092 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:31 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:03.175 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:03.176 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:03.176 INFO:teuthology.orchestra.run.trial130.stdout: Package Arch Version Repo Size 2026-02-21T14:44:03.176 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:03.176 INFO:teuthology.orchestra.run.trial130.stdout:Installing: 2026-02-21T14:44:03.176 INFO:teuthology.orchestra.run.trial130.stdout: librados-devel x86_64 2:20.3.0-5076.g57019f91.el9 ceph 124 k 2026-02-21T14:44:03.176 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:03.177 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:44:03.177 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:03.177 INFO:teuthology.orchestra.run.trial130.stdout:Install 1 Package 2026-02-21T14:44:03.177 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:03.177 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 124 k 2026-02-21T14:44:03.177 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 434 k 2026-02-21T14:44:03.177 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:44:04.220 INFO:teuthology.orchestra.run.trial130.stdout:librados-devel-20.3.0-5076.g57019f91.el9.x86_64 118 kB/s | 124 kB 00:01 2026-02-21T14:44:04.221 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:44:04.221 INFO:teuthology.orchestra.run.trial130.stdout:Total 118 kB/s | 124 kB 00:01 2026-02-21T14:44:04.221 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:44:04.224 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:44:04.224 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:44:04.257 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:44:04.258 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:44:04.313 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:44:04.330 INFO:teuthology.orchestra.run.trial130.stdout: Installing : librados-devel-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:04.410 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: librados-devel-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:04.469 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : librados-devel-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:04.469 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:04.470 INFO:teuthology.orchestra.run.trial130.stdout:Installed: 2026-02-21T14:44:04.470 INFO:teuthology.orchestra.run.trial130.stdout: librados-devel-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:44:04.470 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:04.470 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:04.499 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install libcephfs2 2026-02-21T14:44:04.813 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:32 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:04.877 INFO:teuthology.orchestra.run.trial130.stdout:Package libcephfs2-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:44:04.896 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:04.896 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:44:04.896 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:04.918 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install libcephfs-devel 2026-02-21T14:44:05.235 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:33 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:05.318 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:05.319 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:05.319 INFO:teuthology.orchestra.run.trial130.stdout: Package Arch Version Repo Size 2026-02-21T14:44:05.319 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:05.319 INFO:teuthology.orchestra.run.trial130.stdout:Installing: 2026-02-21T14:44:05.319 INFO:teuthology.orchestra.run.trial130.stdout: libcephfs-devel x86_64 2:20.3.0-5076.g57019f91.el9 ceph 37 k 2026-02-21T14:44:05.320 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:05.320 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:44:05.320 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:05.320 INFO:teuthology.orchestra.run.trial130.stdout:Install 1 Package 2026-02-21T14:44:05.320 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:05.320 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 37 k 2026-02-21T14:44:05.320 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 171 k 2026-02-21T14:44:05.320 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:44:05.677 INFO:teuthology.orchestra.run.trial130.stdout:libcephfs-devel-20.3.0-5076.g57019f91.el9.x86_6 104 kB/s | 37 kB 00:00 2026-02-21T14:44:05.678 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:44:05.678 INFO:teuthology.orchestra.run.trial130.stdout:Total 104 kB/s | 37 kB 00:00 2026-02-21T14:44:05.678 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:44:05.681 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:44:05.681 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:44:05.684 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:44:05.685 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:44:05.705 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:44:05.721 INFO:teuthology.orchestra.run.trial130.stdout: Installing : libcephfs-devel-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:05.780 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:05.839 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : libcephfs-devel-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:05.839 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:05.839 INFO:teuthology.orchestra.run.trial130.stdout:Installed: 2026-02-21T14:44:05.839 INFO:teuthology.orchestra.run.trial130.stdout: libcephfs-devel-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:44:05.840 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:05.840 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:05.870 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install librados2 2026-02-21T14:44:06.180 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:34 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:06.246 INFO:teuthology.orchestra.run.trial130.stdout:Package librados2-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:44:06.264 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:06.265 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:44:06.265 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:06.287 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install librbd1 2026-02-21T14:44:06.592 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:34 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:06.657 INFO:teuthology.orchestra.run.trial130.stdout:Package librbd1-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:44:06.675 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:06.676 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:44:06.676 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:06.696 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install python3-rados 2026-02-21T14:44:07.002 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:35 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:07.066 INFO:teuthology.orchestra.run.trial130.stdout:Package python3-rados-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:44:07.084 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:07.085 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:44:07.085 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:07.106 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install python3-rgw 2026-02-21T14:44:07.420 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:35 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:07.485 INFO:teuthology.orchestra.run.trial130.stdout:Package python3-rgw-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:44:07.503 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:07.504 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:44:07.504 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:07.526 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install python3-cephfs 2026-02-21T14:44:07.839 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:35 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:07.908 INFO:teuthology.orchestra.run.trial130.stdout:Package python3-cephfs-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:44:07.927 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:07.927 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:44:07.927 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:07.949 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install python3-rbd 2026-02-21T14:44:08.261 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:36 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:08.325 INFO:teuthology.orchestra.run.trial130.stdout:Package python3-rbd-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:44:08.344 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:08.344 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:44:08.344 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:08.367 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install rbd-fuse 2026-02-21T14:44:08.682 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:36 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:08.767 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:08.768 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:08.768 INFO:teuthology.orchestra.run.trial130.stdout: Package Architecture Version Repository Size 2026-02-21T14:44:08.768 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:08.768 INFO:teuthology.orchestra.run.trial130.stdout:Installing: 2026-02-21T14:44:08.768 INFO:teuthology.orchestra.run.trial130.stdout: rbd-fuse x86_64 2:20.3.0-5076.g57019f91.el9 ceph 86 k 2026-02-21T14:44:08.768 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:08.768 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:44:08.769 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:08.769 INFO:teuthology.orchestra.run.trial130.stdout:Install 1 Package 2026-02-21T14:44:08.769 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:08.769 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 86 k 2026-02-21T14:44:08.769 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 236 k 2026-02-21T14:44:08.769 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:44:09.196 INFO:teuthology.orchestra.run.trial130.stdout:rbd-fuse-20.3.0-5076.g57019f91.el9.x86_64.rpm 202 kB/s | 86 kB 00:00 2026-02-21T14:44:09.197 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:44:09.197 INFO:teuthology.orchestra.run.trial130.stdout:Total 202 kB/s | 86 kB 00:00 2026-02-21T14:44:09.197 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:44:09.201 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:44:09.201 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:44:09.234 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:44:09.234 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:44:09.286 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:44:09.303 INFO:teuthology.orchestra.run.trial130.stdout: Installing : rbd-fuse-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:09.386 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:09.445 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : rbd-fuse-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:09.445 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:09.446 INFO:teuthology.orchestra.run.trial130.stdout:Installed: 2026-02-21T14:44:09.446 INFO:teuthology.orchestra.run.trial130.stdout: rbd-fuse-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:44:09.446 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:09.446 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:09.474 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install rbd-mirror 2026-02-21T14:44:09.784 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:37 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:09.868 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:09.869 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:09.869 INFO:teuthology.orchestra.run.trial130.stdout: Package Arch Version Repo Size 2026-02-21T14:44:09.869 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:09.869 INFO:teuthology.orchestra.run.trial130.stdout:Installing: 2026-02-21T14:44:09.870 INFO:teuthology.orchestra.run.trial130.stdout: rbd-mirror x86_64 2:20.3.0-5076.g57019f91.el9 ceph 2.6 M 2026-02-21T14:44:09.870 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:09.870 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:44:09.870 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:09.870 INFO:teuthology.orchestra.run.trial130.stdout:Install 1 Package 2026-02-21T14:44:09.870 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:09.870 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 2.6 M 2026-02-21T14:44:09.870 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 11 M 2026-02-21T14:44:09.870 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:44:12.580 INFO:teuthology.orchestra.run.trial130.stdout:rbd-mirror-20.3.0-5076.g57019f91.el9.x86_64.rpm 979 kB/s | 2.6 MB 00:02 2026-02-21T14:44:12.580 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:44:12.580 INFO:teuthology.orchestra.run.trial130.stdout:Total 979 kB/s | 2.6 MB 00:02 2026-02-21T14:44:12.580 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:44:12.585 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:44:12.585 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:44:12.624 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:44:12.624 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:44:12.787 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:44:12.789 INFO:teuthology.orchestra.run.trial130.stdout: Installing : rbd-mirror-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:12.811 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:12.812 INFO:teuthology.orchestra.run.trial130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T14:44:12.812 INFO:teuthology.orchestra.run.trial130.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-21T14:44:12.812 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-21T14:44:12.812 INFO:teuthology.orchestra.run.trial130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-21T14:44:12.812 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:13.137 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : rbd-mirror-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:13.137 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:13.137 INFO:teuthology.orchestra.run.trial130.stdout:Installed: 2026-02-21T14:44:13.138 INFO:teuthology.orchestra.run.trial130.stdout: rbd-mirror-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:44:13.138 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:13.138 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:13.166 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install rbd-nbd 2026-02-21T14:44:13.476 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:41 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:13.560 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:13.560 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:13.560 INFO:teuthology.orchestra.run.trial130.stdout: Package Architecture Version Repository Size 2026-02-21T14:44:13.560 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:13.560 INFO:teuthology.orchestra.run.trial130.stdout:Installing: 2026-02-21T14:44:13.560 INFO:teuthology.orchestra.run.trial130.stdout: rbd-nbd x86_64 2:20.3.0-5076.g57019f91.el9 ceph 166 k 2026-02-21T14:44:13.560 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:13.561 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:44:13.561 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:13.561 INFO:teuthology.orchestra.run.trial130.stdout:Install 1 Package 2026-02-21T14:44:13.561 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:13.561 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 166 k 2026-02-21T14:44:13.561 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 496 k 2026-02-21T14:44:13.561 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:44:14.277 INFO:teuthology.orchestra.run.trial130.stdout:rbd-nbd-20.3.0-5076.g57019f91.el9.x86_64.rpm 232 kB/s | 166 kB 00:00 2026-02-21T14:44:14.277 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:44:14.277 INFO:teuthology.orchestra.run.trial130.stdout:Total 232 kB/s | 166 kB 00:00 2026-02-21T14:44:14.278 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:44:14.282 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:44:14.282 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:44:14.313 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:44:14.314 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:44:14.369 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:44:14.386 INFO:teuthology.orchestra.run.trial130.stdout: Installing : rbd-nbd-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:14.469 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:14.528 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : rbd-nbd-2:20.3.0-5076.g57019f91.el9.x86_64 1/1 2026-02-21T14:44:14.528 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:14.528 INFO:teuthology.orchestra.run.trial130.stdout:Installed: 2026-02-21T14:44:14.528 INFO:teuthology.orchestra.run.trial130.stdout: rbd-nbd-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:44:14.528 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:14.528 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:14.556 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-osd-crimson 2026-02-21T14:44:14.874 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:42 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:14.940 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-osd-crimson-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:44:14.958 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:14.959 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:44:14.959 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:44:14.981 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-21T14:44:15.294 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:12:43 ago on Sat 21 Feb 2026 02:31:32 PM UTC. 2026-02-21T14:44:15.379 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:44:15.380 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:15.380 INFO:teuthology.orchestra.run.trial130.stdout: Package Arch Version Repo Size 2026-02-21T14:44:15.380 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:15.380 INFO:teuthology.orchestra.run.trial130.stdout:Installing: 2026-02-21T14:44:15.380 INFO:teuthology.orchestra.run.trial130.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5076.g57019f91.el9 ceph 429 M 2026-02-21T14:44:15.380 INFO:teuthology.orchestra.run.trial130.stdout:Installing weak dependencies: 2026-02-21T14:44:15.380 INFO:teuthology.orchestra.run.trial130.stdout: ceph-debugsource x86_64 2:20.3.0-5076.g57019f91.el9 ceph 12 M 2026-02-21T14:44:15.380 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:15.380 INFO:teuthology.orchestra.run.trial130.stdout:Transaction Summary 2026-02-21T14:44:15.380 INFO:teuthology.orchestra.run.trial130.stdout:================================================================================ 2026-02-21T14:44:15.381 INFO:teuthology.orchestra.run.trial130.stdout:Install 2 Packages 2026-02-21T14:44:15.381 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:44:15.381 INFO:teuthology.orchestra.run.trial130.stdout:Total download size: 441 M 2026-02-21T14:44:15.381 INFO:teuthology.orchestra.run.trial130.stdout:Installed size: 2.7 G 2026-02-21T14:44:15.381 INFO:teuthology.orchestra.run.trial130.stdout:Downloading Packages: 2026-02-21T14:44:32.274 INFO:teuthology.orchestra.run.trial130.stdout:(1/2): ceph-debugsource-20.3.0-5076.g57019f91.e 728 kB/s | 12 MB 00:16 2026-02-21T14:57:07.797 INFO:teuthology.orchestra.run.trial130.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5076.g 568 kB/s | 429 MB 12:52 2026-02-21T14:57:07.798 INFO:teuthology.orchestra.run.trial130.stdout:-------------------------------------------------------------------------------- 2026-02-21T14:57:07.798 INFO:teuthology.orchestra.run.trial130.stdout:Total 584 kB/s | 441 MB 12:52 2026-02-21T14:57:07.799 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction check 2026-02-21T14:57:07.804 INFO:teuthology.orchestra.run.trial130.stdout:Transaction check succeeded. 2026-02-21T14:57:07.804 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction test 2026-02-21T14:57:09.171 INFO:teuthology.orchestra.run.trial130.stdout:Transaction test succeeded. 2026-02-21T14:57:09.171 INFO:teuthology.orchestra.run.trial130.stdout:Running transaction 2026-02-21T14:57:12.749 INFO:teuthology.orchestra.run.trial130.stdout: Preparing : 1/1 2026-02-21T14:57:35.217 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-debugsource-2:20.3.0-5076.g57019f91.el9.x86_64 1/2 2026-02-21T14:57:35.234 INFO:teuthology.orchestra.run.trial130.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5076.g57019f91.e 2/2 2026-02-21T14:57:37.211 INFO:teuthology.orchestra.run.trial130.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5076.g57019f91.e 2/2 2026-02-21T14:57:37.211 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-debugsource-2:20.3.0-5076.g57019f91.el9.x86_64 1/2 2026-02-21T14:57:37.271 INFO:teuthology.orchestra.run.trial130.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5076.g57019f91.e 2/2 2026-02-21T14:57:37.271 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:57:37.271 INFO:teuthology.orchestra.run.trial130.stdout:Installed: 2026-02-21T14:57:37.271 INFO:teuthology.orchestra.run.trial130.stdout: ceph-debugsource-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:57:37.271 INFO:teuthology.orchestra.run.trial130.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5076.g57019f91.el9.x86_64 2026-02-21T14:57:37.272 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:57:37.272 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:57:37.349 DEBUG:teuthology.parallel:result is None 2026-02-21T14:57:37.350 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=57019f91dea307b3c704ce2392a5f854512b5900 2026-02-21T14:57:37.515 DEBUG:teuthology.orchestra.run.trial130:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-21T14:57:37.535 INFO:teuthology.orchestra.run.trial130.stdout:20.3.0-5076.g57019f91.el9 2026-02-21T14:57:37.536 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5076.g57019f91.el9 2026-02-21T14:57:37.536 INFO:teuthology.task.install:The correct ceph version 20.3.0-5076.g57019f91 is installed. 2026-02-21T14:57:37.539 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-21T14:57:37.539 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:57:37.539 DEBUG:teuthology.orchestra.run.trial130:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-21T14:57:37.603 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-21T14:57:37.603 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:57:37.603 DEBUG:teuthology.orchestra.run.trial130:> sudo dd of=/usr/bin/daemon-helper 2026-02-21T14:57:37.669 DEBUG:teuthology.orchestra.run.trial130:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-21T14:57:37.734 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-21T14:57:37.735 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:57:37.735 DEBUG:teuthology.orchestra.run.trial130:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-21T14:57:37.798 DEBUG:teuthology.orchestra.run.trial130:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-21T14:57:37.861 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-21T14:57:37.862 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:57:37.862 DEBUG:teuthology.orchestra.run.trial130:> sudo dd of=/usr/bin/stdin-killer 2026-02-21T14:57:37.925 DEBUG:teuthology.orchestra.run.trial130:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-21T14:57:37.991 INFO:teuthology.run_tasks:Running task pexec... 2026-02-21T14:57:38.008 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-21T14:57:38.009 DEBUG:teuthology.orchestra.run.trial130:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-21T14:57:38.033 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial130.front.sepia.ceph.com 2026-02-21T14:57:38.034 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-21T14:57:38.084 DEBUG:teuthology.parallel:result is None 2026-02-21T14:57:38.084 INFO:teuthology.run_tasks:Running task install... 2026-02-21T14:57:38.098 DEBUG:teuthology.task.install:project ceph 2026-02-21T14:57:38.099 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '57019f91dea307b3c704ce2392a5f854512b5900'}} 2026-02-21T14:57:38.099 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '57019f91dea307b3c704ce2392a5f854512b5900'} 2026-02-21T14:57:38.099 INFO:teuthology.task.install:Using flavor: debug 2026-02-21T14:57:38.107 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T14:57:38.108 INFO:teuthology.task.install:extra packages: [] 2026-02-21T14:57:38.108 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '57019f91dea307b3c704ce2392a5f854512b5900', 'tag': None, 'wait_for_package': False} 2026-02-21T14:57:38.108 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=57019f91dea307b3c704ce2392a5f854512b5900 2026-02-21T14:57:38.273 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-trans-cc-opt-rbm-drop-brf/57019f91dea307b3c704ce2392a5f854512b5900/centos/9/flavors/debug/ 2026-02-21T14:57:38.273 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5076.g57019f91 2026-02-21T14:57:38.402 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-trans-cc-opt-rbm-drop-brf/57019f91dea307b3c704ce2392a5f854512b5900/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-trans-cc-opt-rbm-drop-brf/57019f91dea307b3c704ce2392a5f854512b5900/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-trans-cc-opt-rbm-drop-brf/57019f91dea307b3c704ce2392a5f854512b5900/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-21T14:57:38.402 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:57:38.402 DEBUG:teuthology.orchestra.run.trial130:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-21T14:57:38.426 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-21T14:57:38.427 DEBUG:teuthology.orchestra.run.trial130:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/57019f91dea307b3c704ce2392a5f854512b5900/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-21T14:57:38.491 DEBUG:teuthology.orchestra.run.trial130:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-21T14:57:38.555 DEBUG:teuthology.orchestra.run.trial130:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-21T14:57:38.612 INFO:teuthology.orchestra.run.trial130.stdout:check_obsoletes = 1 2026-02-21T14:57:38.622 DEBUG:teuthology.orchestra.run.trial130:> sudo yum clean all 2026-02-21T14:57:38.782 INFO:teuthology.orchestra.run.trial130.stdout:56 files removed 2026-02-21T14:57:38.797 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-radosgw 2026-02-21T14:57:39.085 INFO:teuthology.orchestra.run.trial130.stdout:ceph packages for x86_64 691 kB/s | 94 kB 00:00 2026-02-21T14:57:39.240 INFO:teuthology.orchestra.run.trial130.stdout:ceph noarch packages 186 kB/s | 24 kB 00:00 2026-02-21T14:57:39.383 INFO:teuthology.orchestra.run.trial130.stdout:ceph source packages 17 kB/s | 2.1 kB 00:00 2026-02-21T14:57:42.913 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.9 MB 00:03 2026-02-21T14:57:48.142 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 27 MB 00:04 2026-02-21T14:57:51.671 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - CRB 6.8 MB/s | 7.8 MB 00:01 2026-02-21T14:57:52.529 INFO:teuthology.orchestra.run.trial130.stdout:CentOS Stream 9 - Extras packages 145 kB/s | 20 kB 00:00 2026-02-21T14:57:53.068 INFO:teuthology.orchestra.run.trial130.stdout:Extra Packages for Enterprise Linux 43 MB/s | 20 MB 00:00 2026-02-21T14:57:56.315 INFO:teuthology.orchestra.run.trial130.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-21T14:57:57.496 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-radosgw-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:57:57.515 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:57:57.515 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:57:57.515 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:57:57.540 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-test 2026-02-21T14:57:57.853 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:01 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:57:57.917 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-test-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:57:57.935 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:57:57.936 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:57:57.936 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:57:57.958 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph 2026-02-21T14:57:58.266 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:57:58.332 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:57:58.351 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:57:58.351 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:57:58.352 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:57:58.373 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-base 2026-02-21T14:57:58.683 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:57:58.746 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-base-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:57:58.765 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:57:58.766 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:57:58.766 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:57:58.786 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install cephadm 2026-02-21T14:57:59.098 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:03 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:57:59.162 INFO:teuthology.orchestra.run.trial130.stdout:Package cephadm-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:57:59.181 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:57:59.181 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:57:59.181 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:57:59.202 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-immutable-object-cache 2026-02-21T14:57:59.513 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:03 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:57:59.578 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-immutable-object-cache-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:57:59.597 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:57:59.597 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:57:59.597 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:57:59.619 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-mgr 2026-02-21T14:57:59.934 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:03 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:57:59.999 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-mgr-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:00.018 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:00.018 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:00.018 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:00.039 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-mgr-dashboard 2026-02-21T14:58:00.349 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:04 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:00.414 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-mgr-dashboard-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:58:00.432 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:00.433 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:00.433 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:00.455 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-21T14:58:00.767 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:04 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:00.832 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:58:00.850 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:00.851 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:00.851 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:00.873 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-mgr-rook 2026-02-21T14:58:01.188 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:05 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:01.253 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-mgr-rook-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:58:01.271 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:01.272 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:01.272 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:01.295 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-mgr-cephadm 2026-02-21T14:58:01.605 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:05 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:01.670 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-mgr-cephadm-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:58:01.688 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:01.689 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:01.689 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:01.710 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-osd 2026-02-21T14:58:02.025 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:06 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:02.089 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-osd-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:02.107 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:02.108 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:02.108 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:02.130 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-osd-classic 2026-02-21T14:58:02.444 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:06 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:02.509 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-osd-classic-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:02.528 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:02.529 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:02.529 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:02.552 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-fuse 2026-02-21T14:58:02.867 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:06 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:02.930 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-fuse-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:02.949 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:02.950 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:02.950 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:02.971 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install ceph-volume 2026-02-21T14:58:03.284 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:07 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:03.350 INFO:teuthology.orchestra.run.trial130.stdout:Package ceph-volume-2:20.3.0-5076.g57019f91.el9.noarch is already installed. 2026-02-21T14:58:03.368 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:03.369 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:03.369 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:03.390 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install librados-devel 2026-02-21T14:58:03.707 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:07 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:03.772 INFO:teuthology.orchestra.run.trial130.stdout:Package librados-devel-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:03.791 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:03.791 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:03.791 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:03.813 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install libcephfs2 2026-02-21T14:58:04.125 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:08 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:04.191 INFO:teuthology.orchestra.run.trial130.stdout:Package libcephfs2-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:04.210 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:04.211 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:04.211 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:04.232 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install libcephfs-devel 2026-02-21T14:58:04.541 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:08 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:04.605 INFO:teuthology.orchestra.run.trial130.stdout:Package libcephfs-devel-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:04.624 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:04.625 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:04.625 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:04.647 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install librados2 2026-02-21T14:58:04.962 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:08 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:05.026 INFO:teuthology.orchestra.run.trial130.stdout:Package librados2-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:05.045 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:05.045 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:05.045 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:05.067 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install librbd1 2026-02-21T14:58:05.382 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:09 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:05.446 INFO:teuthology.orchestra.run.trial130.stdout:Package librbd1-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:05.465 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:05.465 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:05.466 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:05.488 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install python3-rados 2026-02-21T14:58:05.801 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:09 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:05.866 INFO:teuthology.orchestra.run.trial130.stdout:Package python3-rados-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:05.884 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:05.885 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:05.885 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:05.907 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install python3-rgw 2026-02-21T14:58:06.215 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:10 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:06.280 INFO:teuthology.orchestra.run.trial130.stdout:Package python3-rgw-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:06.298 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:06.299 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:06.299 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:06.321 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install python3-cephfs 2026-02-21T14:58:06.628 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:10 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:06.693 INFO:teuthology.orchestra.run.trial130.stdout:Package python3-cephfs-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:06.712 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:06.713 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:06.713 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:06.735 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install python3-rbd 2026-02-21T14:58:07.050 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:11 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:07.117 INFO:teuthology.orchestra.run.trial130.stdout:Package python3-rbd-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:07.136 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:07.136 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:07.136 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:07.157 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install rbd-fuse 2026-02-21T14:58:07.475 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:11 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:07.539 INFO:teuthology.orchestra.run.trial130.stdout:Package rbd-fuse-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:07.558 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:07.559 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:07.559 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:07.581 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install rbd-mirror 2026-02-21T14:58:07.894 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:11 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:07.959 INFO:teuthology.orchestra.run.trial130.stdout:Package rbd-mirror-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:07.977 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:07.978 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:07.978 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:08.000 DEBUG:teuthology.orchestra.run.trial130:> sudo yum -y install rbd-nbd 2026-02-21T14:58:08.321 INFO:teuthology.orchestra.run.trial130.stdout:Last metadata expiration check: 0:00:12 ago on Sat 21 Feb 2026 02:57:56 PM UTC. 2026-02-21T14:58:08.386 INFO:teuthology.orchestra.run.trial130.stdout:Package rbd-nbd-2:20.3.0-5076.g57019f91.el9.x86_64 is already installed. 2026-02-21T14:58:08.404 INFO:teuthology.orchestra.run.trial130.stdout:Dependencies resolved. 2026-02-21T14:58:08.405 INFO:teuthology.orchestra.run.trial130.stdout:Nothing to do. 2026-02-21T14:58:08.405 INFO:teuthology.orchestra.run.trial130.stdout:Complete! 2026-02-21T14:58:08.431 DEBUG:teuthology.parallel:result is None 2026-02-21T14:58:08.431 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=57019f91dea307b3c704ce2392a5f854512b5900 2026-02-21T14:58:08.597 DEBUG:teuthology.orchestra.run.trial130:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-21T14:58:08.618 INFO:teuthology.orchestra.run.trial130.stdout:20.3.0-5076.g57019f91.el9 2026-02-21T14:58:08.618 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5076.g57019f91.el9 2026-02-21T14:58:08.618 INFO:teuthology.task.install:The correct ceph version 20.3.0-5076.g57019f91 is installed. 2026-02-21T14:58:08.620 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-21T14:58:08.620 INFO:teuthology.run_tasks:Running task ceph... 2026-02-21T14:58:08.715 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-21T14:58:08.715 DEBUG:teuthology.orchestra.run.trial130:> sudo chmod 777 /var/log/ceph 2026-02-21T14:58:08.740 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-21T14:58:08.741 DEBUG:teuthology.orchestra.run.trial130:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-21T14:58:08.804 INFO:tasks.ceph:Creating extra log directories... 2026-02-21T14:58:08.804 DEBUG:teuthology.orchestra.run.trial130:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-21T14:58:08.879 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-21T14:58:08.879 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 25, 'debug seastore backref': 20, 'debug seastore cache': 25, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 25, 'debug seastore odata': 25, 'debug seastore omap': 25, 'debug seastore onode': 25, 'debug seastore t': 25, 'debug seastore tm': 25, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-21T14:58:08.879 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/xuxuehan-2026-02-21_14:23:14-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62724', 'branch': 'wip-trans-cc-opt-rbm-drop-brf', 'description': 'crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} tasks/rbd_lock_and_fence}', 'email': 'xxhdx1985126@gmail.com', 'first_in_suite': False, 'flavor': 'debug', 'job_id': '62724', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'xuxuehan-2026-02-21_14:23:14-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-trans-cc-opt-rbm-drop-brf'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 25, 'debug seastore backref': 20, 'debug seastore cache': 25, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 25, 'debug seastore odata': 25, 'debug seastore omap': 25, 'debug seastore onode': 25, 'debug seastore t': 25, 'debug seastore tm': 25, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '57019f91dea307b3c704ce2392a5f854512b5900'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '57019f91dea307b3c704ce2392a5f854512b5900'}}, 'workunit': {'branch': 'wip-seastore-qa-log-level-up', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f'}}, 'owner': 'scheduled_xuxuehan@soko04.front.sepia.ceph.com', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 2477, 'sha1': '57019f91dea307b3c704ce2392a5f854512b5900', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'wip-seastore-qa-log-level-up', 'suite_path': '/home/teuthworker/src/github.com_xxhdx1985126_ceph_c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/xxhdx1985126/ceph.git', 'suite_sha1': 'c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f', 'targets': {'trial130.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGZX6TDggcgyxvoWI80vWWs3SiZGCI2rO9OX45Gc0UEkzgZVmZXcqujSs4uc/rIhUkF9dqQg6aUenr/qMTo4jwM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '57019f91dea307b3c704ce2392a5f854512b5900'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 25, 'debug seastore backref': 20, 'debug seastore cache': 25, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 25, 'debug seastore odata': 25, 'debug seastore omap': 25, 'debug seastore onode': 25, 'debug seastore t': 25, 'debug seastore tm': 25, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '57019f91dea307b3c704ce2392a5f854512b5900', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rbd/test_lock_fence.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-21_14:23:14', 'tube': 'trial', 'user': 'xuxuehan', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-21T14:58:08.880 DEBUG:teuthology.orchestra.run.trial130:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-21T14:58:08.926 DEBUG:teuthology.orchestra.run.trial130:> sudo install -d -m0777 -- /var/run/ceph 2026-02-21T14:58:08.990 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:08.990 DEBUG:teuthology.orchestra.run.trial130:> dd if=/scratch_devs of=/dev/stdout 2026-02-21T14:58:09.047 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-21T14:58:09.047 DEBUG:teuthology.orchestra.run.trial130:> stat /dev/vg_nvme/lv_1 2026-02-21T14:58:09.104 INFO:teuthology.orchestra.run.trial130.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-21T14:58:09.105 INFO:teuthology.orchestra.run.trial130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T14:58:09.105 INFO:teuthology.orchestra.run.trial130.stdout:Device: 6h/6d Inode: 1304 Links: 1 2026-02-21T14:58:09.105 INFO:teuthology.orchestra.run.trial130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T14:58:09.105 INFO:teuthology.orchestra.run.trial130.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T14:58:09.105 INFO:teuthology.orchestra.run.trial130.stdout:Access: 2026-02-21 14:44:12.967971299 +0000 2026-02-21T14:58:09.105 INFO:teuthology.orchestra.run.trial130.stdout:Modify: 2026-02-21 14:32:27.054925230 +0000 2026-02-21T14:58:09.105 INFO:teuthology.orchestra.run.trial130.stdout:Change: 2026-02-21 14:32:27.054925230 +0000 2026-02-21T14:58:09.105 INFO:teuthology.orchestra.run.trial130.stdout: Birth: 2026-02-21 14:32:27.054925230 +0000 2026-02-21T14:58:09.106 DEBUG:teuthology.orchestra.run.trial130:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-21T14:58:09.168 INFO:teuthology.orchestra.run.trial130.stderr:1+0 records in 2026-02-21T14:58:09.168 INFO:teuthology.orchestra.run.trial130.stderr:1+0 records out 2026-02-21T14:58:09.168 INFO:teuthology.orchestra.run.trial130.stderr:512 bytes copied, 0.000180156 s, 2.8 MB/s 2026-02-21T14:58:09.169 DEBUG:teuthology.orchestra.run.trial130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-21T14:58:09.226 DEBUG:teuthology.orchestra.run.trial130:> stat /dev/vg_nvme/lv_2 2026-02-21T14:58:09.283 INFO:teuthology.orchestra.run.trial130.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-21T14:58:09.283 INFO:teuthology.orchestra.run.trial130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T14:58:09.283 INFO:teuthology.orchestra.run.trial130.stdout:Device: 6h/6d Inode: 1294 Links: 1 2026-02-21T14:58:09.283 INFO:teuthology.orchestra.run.trial130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T14:58:09.283 INFO:teuthology.orchestra.run.trial130.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T14:58:09.283 INFO:teuthology.orchestra.run.trial130.stdout:Access: 2026-02-21 14:44:12.967971299 +0000 2026-02-21T14:58:09.283 INFO:teuthology.orchestra.run.trial130.stdout:Modify: 2026-02-21 14:32:27.053925268 +0000 2026-02-21T14:58:09.283 INFO:teuthology.orchestra.run.trial130.stdout:Change: 2026-02-21 14:32:27.054925230 +0000 2026-02-21T14:58:09.284 INFO:teuthology.orchestra.run.trial130.stdout: Birth: 2026-02-21 14:32:27.053925268 +0000 2026-02-21T14:58:09.284 DEBUG:teuthology.orchestra.run.trial130:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-21T14:58:09.347 INFO:teuthology.orchestra.run.trial130.stderr:1+0 records in 2026-02-21T14:58:09.347 INFO:teuthology.orchestra.run.trial130.stderr:1+0 records out 2026-02-21T14:58:09.347 INFO:teuthology.orchestra.run.trial130.stderr:512 bytes copied, 0.000133849 s, 3.8 MB/s 2026-02-21T14:58:09.349 DEBUG:teuthology.orchestra.run.trial130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-21T14:58:09.406 DEBUG:teuthology.orchestra.run.trial130:> stat /dev/vg_nvme/lv_3 2026-02-21T14:58:09.462 INFO:teuthology.orchestra.run.trial130.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-21T14:58:09.462 INFO:teuthology.orchestra.run.trial130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T14:58:09.462 INFO:teuthology.orchestra.run.trial130.stdout:Device: 6h/6d Inode: 1292 Links: 1 2026-02-21T14:58:09.462 INFO:teuthology.orchestra.run.trial130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T14:58:09.462 INFO:teuthology.orchestra.run.trial130.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T14:58:09.462 INFO:teuthology.orchestra.run.trial130.stdout:Access: 2026-02-21 14:44:12.968971291 +0000 2026-02-21T14:58:09.462 INFO:teuthology.orchestra.run.trial130.stdout:Modify: 2026-02-21 14:32:27.053925268 +0000 2026-02-21T14:58:09.462 INFO:teuthology.orchestra.run.trial130.stdout:Change: 2026-02-21 14:32:27.053925268 +0000 2026-02-21T14:58:09.463 INFO:teuthology.orchestra.run.trial130.stdout: Birth: 2026-02-21 14:32:27.053925268 +0000 2026-02-21T14:58:09.463 DEBUG:teuthology.orchestra.run.trial130:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-21T14:58:09.524 INFO:teuthology.orchestra.run.trial130.stderr:1+0 records in 2026-02-21T14:58:09.524 INFO:teuthology.orchestra.run.trial130.stderr:1+0 records out 2026-02-21T14:58:09.524 INFO:teuthology.orchestra.run.trial130.stderr:512 bytes copied, 0.000147895 s, 3.5 MB/s 2026-02-21T14:58:09.526 DEBUG:teuthology.orchestra.run.trial130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-21T14:58:09.583 DEBUG:teuthology.orchestra.run.trial130:> stat /dev/vg_nvme/lv_4 2026-02-21T14:58:09.640 INFO:teuthology.orchestra.run.trial130.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-21T14:58:09.640 INFO:teuthology.orchestra.run.trial130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T14:58:09.641 INFO:teuthology.orchestra.run.trial130.stdout:Device: 6h/6d Inode: 1299 Links: 1 2026-02-21T14:58:09.641 INFO:teuthology.orchestra.run.trial130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T14:58:09.641 INFO:teuthology.orchestra.run.trial130.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T14:58:09.641 INFO:teuthology.orchestra.run.trial130.stdout:Access: 2026-02-21 14:44:12.968971291 +0000 2026-02-21T14:58:09.641 INFO:teuthology.orchestra.run.trial130.stdout:Modify: 2026-02-21 14:32:27.054925230 +0000 2026-02-21T14:58:09.641 INFO:teuthology.orchestra.run.trial130.stdout:Change: 2026-02-21 14:32:27.054925230 +0000 2026-02-21T14:58:09.641 INFO:teuthology.orchestra.run.trial130.stdout: Birth: 2026-02-21 14:32:27.054925230 +0000 2026-02-21T14:58:09.641 DEBUG:teuthology.orchestra.run.trial130:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-21T14:58:09.703 INFO:teuthology.orchestra.run.trial130.stderr:1+0 records in 2026-02-21T14:58:09.704 INFO:teuthology.orchestra.run.trial130.stderr:1+0 records out 2026-02-21T14:58:09.704 INFO:teuthology.orchestra.run.trial130.stderr:512 bytes copied, 0.000144489 s, 3.5 MB/s 2026-02-21T14:58:09.705 DEBUG:teuthology.orchestra.run.trial130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-21T14:58:09.763 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T14:58:09.764 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial130.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-21T14:58:09.764 INFO:tasks.ceph:Generating config... 2026-02-21T14:58:09.766 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-21T14:58:09.766 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-21T14:58:09.766 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-21T14:58:09.766 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-21T14:58:09.767 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-21T14:58:09.767 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-21T14:58:09.767 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-21T14:58:09.767 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-21T14:58:09.767 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-21T14:58:09.767 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-21T14:58:09.767 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-21T14:58:09.767 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-21T14:58:09.767 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-21T14:58:09.767 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-21T14:58:09.767 INFO:tasks.ceph:[osd] debug seastore = 25 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore cache = 25 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore lba = 25 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore odata = 25 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore omap = 25 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore onode = 25 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore t = 25 2026-02-21T14:58:09.768 INFO:tasks.ceph:[osd] debug seastore tm = 25 2026-02-21T14:58:09.769 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-21T14:58:09.769 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-21T14:58:09.769 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-21T14:58:09.769 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-21T14:58:09.769 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-21T14:58:09.769 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-21T14:58:09.769 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-21T14:58:09.769 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-21T14:58:09.769 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-21T14:58:09.769 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-21T14:58:09.769 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-21T14:58:09.769 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-21T14:58:09.770 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-21T14:58:09.770 INFO:tasks.ceph:Setting up mon.a... 2026-02-21T14:58:09.770 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-21T14:58:09.845 INFO:teuthology.orchestra.run.trial130.stdout:creating /etc/ceph/ceph.keyring 2026-02-21T14:58:09.847 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-21T14:58:09.933 DEBUG:teuthology.orchestra.run.trial130:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T14:58:09.956 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.130')] 2026-02-21T14:58:09.957 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.130', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 25, 'debug seastore backref': 20, 'debug seastore cache': 25, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 25, 'debug seastore odata': 25, 'debug seastore omap': 25, 'debug seastore onode': 25, 'debug seastore t': 25, 'debug seastore tm': 25, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-21T14:58:09.959 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:09.959 DEBUG:teuthology.orchestra.run.trial130:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T14:58:10.015 DEBUG:teuthology.orchestra.run.trial130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.130 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-21T14:58:10.094 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:10.102+0000 7f5195a2cf40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:10.094 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:10.102+0000 7f5195a2cf40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:10.094 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:10.102+0000 7f5195a2cf40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:10.095 INFO:teuthology.orchestra.run.trial130.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-21T14:58:10.095 INFO:teuthology.orchestra.run.trial130.stdout:monmaptool: generated fsid 05ac4a4e-7e84-4852-9cff-af5d19cabadf 2026-02-21T14:58:10.095 INFO:teuthology.orchestra.run.trial130.stdout:setting min_mon_release = tentacle 2026-02-21T14:58:10.095 INFO:teuthology.orchestra.run.trial130.stdout:epoch 0 2026-02-21T14:58:10.095 INFO:teuthology.orchestra.run.trial130.stdout:fsid 05ac4a4e-7e84-4852-9cff-af5d19cabadf 2026-02-21T14:58:10.095 INFO:teuthology.orchestra.run.trial130.stdout:last_changed 2026-02-21T14:58:10.103538+0000 2026-02-21T14:58:10.096 INFO:teuthology.orchestra.run.trial130.stdout:created 2026-02-21T14:58:10.103538+0000 2026-02-21T14:58:10.096 INFO:teuthology.orchestra.run.trial130.stdout:min_mon_release 20 (tentacle) 2026-02-21T14:58:10.096 INFO:teuthology.orchestra.run.trial130.stdout:election_strategy: 1 2026-02-21T14:58:10.096 INFO:teuthology.orchestra.run.trial130.stdout:0: [v2:10.20.193.130:3300/0,v1:10.20.193.130:6789/0] mon.a 2026-02-21T14:58:10.096 INFO:teuthology.orchestra.run.trial130.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-21T14:58:10.097 DEBUG:teuthology.orchestra.run.trial130:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T14:58:10.154 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 05ac4a4e-7e84-4852-9cff-af5d19cabadf... 2026-02-21T14:58:10.155 DEBUG:teuthology.orchestra.run.trial130:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-21T14:58:10.234 INFO:teuthology.orchestra.run.trial130.stdout:[global] 2026-02-21T14:58:10.235 INFO:teuthology.orchestra.run.trial130.stdout: chdir = "" 2026-02-21T14:58:10.235 INFO:teuthology.orchestra.run.trial130.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-21T14:58:10.235 INFO:teuthology.orchestra.run.trial130.stdout: auth supported = cephx 2026-02-21T14:58:10.235 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.235 INFO:teuthology.orchestra.run.trial130.stdout: filestore xattr use omap = true 2026-02-21T14:58:10.235 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.235 INFO:teuthology.orchestra.run.trial130.stdout: mon clock drift allowed = 1.000 2026-02-21T14:58:10.235 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.235 INFO:teuthology.orchestra.run.trial130.stdout: osd crush chooseleaf type = 0 2026-02-21T14:58:10.235 INFO:teuthology.orchestra.run.trial130.stdout: auth debug = true 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: ms die on old message = true 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: ms die on bug = true 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: mon pg warn max object skew = 0 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: # disable pg_autoscaler by default for new pools 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: osd pool default size = 2 2026-02-21T14:58:10.236 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: mon osd allow primary affinity = true 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: mon osd allow pg remap = true 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: mon warn on legacy crush tunables = false 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: mon warn on crush straw calc version zero = false 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: mon warn on no sortbitwise = false 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: mon warn on osd down out interval zero = false 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: mon warn on too few osds = false 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: mon_allow_pool_size_one = true 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.237 INFO:teuthology.orchestra.run.trial130.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: osd default data pool replay window = 5 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: mon allow pool delete = true 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: mon cluster log file level = debug 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: debug asserts on shutdown = true 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: mon health detail to clog = false 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: mon host = 10.20.193.130 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: ms client mode = crc 2026-02-21T14:58:10.238 INFO:teuthology.orchestra.run.trial130.stdout: ms cluster mode = crc 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: ms mon client mode = crc 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: ms mon cluster mode = crc 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: ms mon service mode = crc 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: ms service mode = crc 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: fsid = 05ac4a4e-7e84-4852-9cff-af5d19cabadf 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout:[osd] 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: osd journal size = 100 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: osd scrub load threshold = 5.0 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: osd scrub max interval = 600 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: osd mclock profile = high_recovery_ops 2026-02-21T14:58:10.239 INFO:teuthology.orchestra.run.trial130.stdout: osd mclock skip benchmark = true 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: osd recover clone overlap = true 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: osd recovery max chunk = 1048576 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: osd debug shutdown = true 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: osd debug op order = true 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: osd debug verify stray on activate = true 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: osd debug trim objects = true 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: osd open classes on start = true 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: osd debug pg log writeout = true 2026-02-21T14:58:10.240 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: osd deep scrub update digest min age = 30 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: osd map max advance = 10 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: journal zero on create = true 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: filestore ondisk finisher threads = 3 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: filestore apply finisher threads = 3 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: bdev debug aio = true 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: osd debug misdirected ops = true 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: debug monc = 20 2026-02-21T14:58:10.241 INFO:teuthology.orchestra.run.trial130.stdout: crimson cpu num = 4 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: crimson osd obc lru size = 10 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: debug ms = 20 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: debug osd = 20 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore = 25 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore backref = 20 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore cache = 25 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore cleaner = 20 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore device = 20 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore epm = 20 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore fixedkv tree = 20 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore journal = 20 2026-02-21T14:58:10.242 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore lba = 25 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore odata = 25 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore omap = 25 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore onode = 25 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore t = 25 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: debug seastore tm = 25 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: osd objectstore = seastore 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: osd shutdown pgref assert = True 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: seastore cachepin size pershard = 64M 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: seastore max concurrent transactions = 8 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout:[mgr] 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: debug ms = 1 2026-02-21T14:58:10.243 INFO:teuthology.orchestra.run.trial130.stdout: debug mgr = 20 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: debug mon = 20 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: debug auth = 20 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: mon reweight min pgs per osd = 4 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: mon reweight min bytes per osd = 10 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: mgr/telemetry/nag = false 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout:[mon] 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: debug ms = 1 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: debug mon = 20 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: debug paxos = 20 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: debug auth = 20 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: mon data avail warn = 5 2026-02-21T14:58:10.244 INFO:teuthology.orchestra.run.trial130.stdout: mon mgr mkfs grace = 240 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: mon reweight min pgs per osd = 4 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: mon osd reporter subtree level = osd 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: mon osd prime pg temp = true 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: mon reweight min bytes per osd = 10 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: auth mon ticket ttl = 660 # 11m 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: auth service ticket ttl = 240 # 4m 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: # don't complain about insecure global_id in the test suite 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-21T14:58:10.245 INFO:teuthology.orchestra.run.trial130.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: # 1m isn't quite enough 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: mon_down_mkfs_grace = 2m 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: mon_warn_on_filestore_osds = false 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: mon min osdmap epochs = 50 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: mon osdmap full prune interval = 2 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: mon osdmap full prune min = 15 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: mon osdmap full prune txsize = 2 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: paxos service trim min = 10 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: osd pool default crimson = True 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout:[client] 2026-02-21T14:58:10.246 INFO:teuthology.orchestra.run.trial130.stdout: rgw cache enabled = true 2026-02-21T14:58:10.247 INFO:teuthology.orchestra.run.trial130.stdout: rgw enable ops log = true 2026-02-21T14:58:10.247 INFO:teuthology.orchestra.run.trial130.stdout: rgw enable usage log = true 2026-02-21T14:58:10.247 INFO:teuthology.orchestra.run.trial130.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-21T14:58:10.247 INFO:teuthology.orchestra.run.trial130.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-21T14:58:10.247 INFO:teuthology.orchestra.run.trial130.stdout:[mon.a] 2026-02-21T14:58:10.248 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-21T14:58:10.248 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-21T14:58:10.339 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-21T14:58:10.339 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:10.339 DEBUG:teuthology.orchestra.run.trial130:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-21T14:58:10.355 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:10.356 DEBUG:teuthology.orchestra.run.trial130:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-21T14:58:10.412 INFO:tasks.ceph:Sending monmap to node ubuntu@trial130.front.sepia.ceph.com 2026-02-21T14:58:10.413 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:10.413 DEBUG:teuthology.orchestra.run.trial130:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-21T14:58:10.413 DEBUG:teuthology.orchestra.run.trial130:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T14:58:10.488 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:10.488 DEBUG:teuthology.orchestra.run.trial130:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-21T14:58:10.545 INFO:tasks.ceph:Setting up mon nodes... 2026-02-21T14:58:10.545 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-21T14:58:10.546 DEBUG:teuthology.orchestra.run.trial130:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T14:58:10.633 INFO:teuthology.orchestra.run.trial130.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T14:58:10.636 INFO:tasks.ceph:Setting up mds nodes... 2026-02-21T14:58:10.637 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-21T14:58:10.637 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-21T14:58:10.678 INFO:teuthology.orchestra.run.trial130.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-21T14:58:10.690 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-21T14:58:10.690 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial130.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-21T14:58:10.690 DEBUG:teuthology.orchestra.run.trial130:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-21T14:58:10.754 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T14:58:10.754 INFO:tasks.ceph:role: osd.0 2026-02-21T14:58:10.754 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial130.front.sepia.ceph.com 2026-02-21T14:58:10.754 DEBUG:teuthology.orchestra.run.trial130:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-21T14:58:10.819 INFO:teuthology.orchestra.run.trial130.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T14:58:10.819 INFO:teuthology.orchestra.run.trial130.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T14:58:10.819 INFO:teuthology.orchestra.run.trial130.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T14:58:10.819 INFO:teuthology.orchestra.run.trial130.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T14:58:10.819 INFO:teuthology.orchestra.run.trial130.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T14:58:10.819 INFO:teuthology.orchestra.run.trial130.stdout: = sunit=0 swidth=0 blks 2026-02-21T14:58:10.819 INFO:teuthology.orchestra.run.trial130.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T14:58:10.820 INFO:teuthology.orchestra.run.trial130.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T14:58:10.820 INFO:teuthology.orchestra.run.trial130.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T14:58:10.820 INFO:teuthology.orchestra.run.trial130.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T14:58:11.440 INFO:teuthology.orchestra.run.trial130.stdout:Discarding blocks...Done. 2026-02-21T14:58:11.442 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial130.front.sepia.ceph.com -o noatime 2026-02-21T14:58:11.442 DEBUG:teuthology.orchestra.run.trial130:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-21T14:58:11.510 DEBUG:teuthology.orchestra.run.trial130:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-21T14:58:11.577 DEBUG:teuthology.orchestra.run.trial130:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-21T14:58:11.643 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T14:58:11.643 INFO:tasks.ceph:role: osd.1 2026-02-21T14:58:11.643 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial130.front.sepia.ceph.com 2026-02-21T14:58:11.643 DEBUG:teuthology.orchestra.run.trial130:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-21T14:58:11.707 INFO:teuthology.orchestra.run.trial130.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T14:58:11.708 INFO:teuthology.orchestra.run.trial130.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T14:58:11.708 INFO:teuthology.orchestra.run.trial130.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T14:58:11.708 INFO:teuthology.orchestra.run.trial130.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T14:58:11.708 INFO:teuthology.orchestra.run.trial130.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T14:58:11.708 INFO:teuthology.orchestra.run.trial130.stdout: = sunit=0 swidth=0 blks 2026-02-21T14:58:11.708 INFO:teuthology.orchestra.run.trial130.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T14:58:11.708 INFO:teuthology.orchestra.run.trial130.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T14:58:11.708 INFO:teuthology.orchestra.run.trial130.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T14:58:11.708 INFO:teuthology.orchestra.run.trial130.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T14:58:12.264 INFO:teuthology.orchestra.run.trial130.stdout:Discarding blocks...Done. 2026-02-21T14:58:12.267 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial130.front.sepia.ceph.com -o noatime 2026-02-21T14:58:12.267 DEBUG:teuthology.orchestra.run.trial130:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-21T14:58:12.339 DEBUG:teuthology.orchestra.run.trial130:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-21T14:58:12.409 DEBUG:teuthology.orchestra.run.trial130:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-21T14:58:12.474 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T14:58:12.474 INFO:tasks.ceph:role: osd.2 2026-02-21T14:58:12.474 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial130.front.sepia.ceph.com 2026-02-21T14:58:12.474 DEBUG:teuthology.orchestra.run.trial130:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-21T14:58:12.540 INFO:teuthology.orchestra.run.trial130.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T14:58:12.540 INFO:teuthology.orchestra.run.trial130.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T14:58:12.540 INFO:teuthology.orchestra.run.trial130.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T14:58:12.540 INFO:teuthology.orchestra.run.trial130.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T14:58:12.540 INFO:teuthology.orchestra.run.trial130.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T14:58:12.540 INFO:teuthology.orchestra.run.trial130.stdout: = sunit=0 swidth=0 blks 2026-02-21T14:58:12.540 INFO:teuthology.orchestra.run.trial130.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T14:58:12.541 INFO:teuthology.orchestra.run.trial130.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T14:58:12.541 INFO:teuthology.orchestra.run.trial130.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T14:58:12.541 INFO:teuthology.orchestra.run.trial130.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T14:58:13.151 INFO:teuthology.orchestra.run.trial130.stdout:Discarding blocks...Done. 2026-02-21T14:58:13.153 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial130.front.sepia.ceph.com -o noatime 2026-02-21T14:58:13.154 DEBUG:teuthology.orchestra.run.trial130:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-21T14:58:13.222 DEBUG:teuthology.orchestra.run.trial130:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-21T14:58:13.289 DEBUG:teuthology.orchestra.run.trial130:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T14:58:13.440 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T14:58:13.496 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:13,505 seastar - Reactor backend: linux-aio 2026-02-21T14:58:13.513 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:13,522 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:13.514 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:13,523 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:13.516 INFO:teuthology.orchestra.run.trial130.stderr:==39061==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T14:58:13.570 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:13,579 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-21T14:58:13.570 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:13,579 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-21T14:58:13.570 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:13,579 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-21T14:58:13.570 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:13,579 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-21T14:58:13.571 INFO:teuthology.orchestra.run.trial130.stderr:DEBUG 2026-02-21 14:58:13,579 [shard 0:main] seastar - reactor::drain 2026-02-21T14:58:15.613 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:15.613 INFO:teuthology.orchestra.run.trial130.stderr:Suppressions used: 2026-02-21T14:58:15.613 INFO:teuthology.orchestra.run.trial130.stderr: count bytes template 2026-02-21T14:58:15.614 INFO:teuthology.orchestra.run.trial130.stderr: 1 8 MallocExtension::Register 2026-02-21T14:58:15.614 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:15.614 INFO:teuthology.orchestra.run.trial130.stderr: 2026-02-21T14:58:15.632 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T14:58:15.632 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T14:58:15.675 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T14:58:15.676 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T14:58:15.682 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:15,691 seastar - Reactor backend: linux-aio 2026-02-21T14:58:15.693 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:15,702 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:15.693 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:15,702 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:15.696 INFO:teuthology.orchestra.run.trial130.stderr:==39059==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T14:58:15.696 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:15,706 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T14:58:15.754 INFO:teuthology.orchestra.run.trial130.stderr:DEBUG 2026-02-21 14:58:15,763 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-21T14:58:15.834 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c6 /lib64/libasan.so.8+0x15197 0x1126530 0xf2ec95 0x9d51d7d 0x9d5236a 0x9d526ed 0x9d62844 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f927ce 0x6f965cf 0x6f9c179 0x74102d6 0x743293e 0x7282937 0x7282a2f 0x7353ce2 0x7353d9b 0x7353f52 0x74329bf 0x7282aac 0x7282b31 0x72ae77b 0x741b166 0x741b1eb 0x741b2c9 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:15.834 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x1126530 0xf2ec95 0x9c6ef73 0x9c6f43d 0x9c6f6b0 0x9c6f966 0x9c6ff28 0x9c783cf 0x9c78aaf 0x9d615fc 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f927ce 0x6f965cf 0x6f9c179 0x74102d6 0x743293e 0x7282937 0x7282a2f 0x7353ce2 0x7353d9b 0x7353f52 0x74329bf 0x7282aac 0x7282b31 0x72ae77b 0x741b166 0x741b1eb 0x741b2c9 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:15.834 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1523c 0x1126530 0xf2ec95 0x9c6f140 0x9c6f43d 0x9c6f6b0 0x9c713d7 0x9c71924 0x9c71b23 0x9d56e45 0x9d575a0 0x9d6182e 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f927ce 0x6f965cf 0x6f9c179 0x74102d6 0x743293e 0x7282937 0x7282a2f 0x7353ce2 0x7353d9b 0x7353f52 0x74329bf 0x7282aac 0x7282b31 0x72ae77b 0x741b166 0x741b1eb 0x741b2c9 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:15.866 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb08f6 /lib64/libasan.so.8+0xb1437 0xb6a317 0xe3cebb 0x315b6b2 0x315e88f 0x3354f03 0x3386a89 0x3387175 0x33df66d 0x33e0504 0xd01deb 0xd0209e 0xe3bc45 0x6f6009a 0x71942bf 0x7112154 0x6f935ba 0x754ac4a 0x754ad9e 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9f98c1c 0x9c16821 0x9c1705c 0xecfd0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47b04 2026-02-21T14:58:15.933 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a5230f 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9f98c1c 0x9c16821 0x9c1705c 0xecfd0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47b04 2026-02-21T14:58:15.970 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a511d0 0x7d8f268 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:15.971 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x2a5230f 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:15.971 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a510f7 0x7d8f268 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:16.098 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe1e15b 0x9d673eb 0x789f8df 0x7bb1504 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f3661e 0x6f38986 0x747af76 0x7299edd 0x7299fdf 0x729a0d7 0x7352eae 0x7352f67 0x735311a 0x7353252 0x73532cd 0x7353352 0x73534ad 0x741abe3 0x747afff 0x7281372 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:16.098 INFO:teuthology.orchestra.run.trial130.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff9ba75c69 0xffffffff9bc1f8c7 0xffffffff9bc1fb20 0xffffffff9ba9525e 0xffffffff9bde61a9 0xffffffff9bdc9715 0xffffffff9bdd3aeb 0xffffffff9bdd3f4b 0xffffffff9ba910d2 0xffffffff9c74d451 0xffffffff9c800ba2 2026-02-21T14:58:16.098 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x10917f5 0xf2eca0 0x9c6f140 0x9c6f43d 0x9c6f6b0 0x9c6f966 0x9c6ff28 0x9c783cf 0x9c78aaf 0x9d615fc 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f3661e 0x6f38986 0x747af76 0x7299edd 0x7299fdf 0x729a0d7 0x7352eae 0x7352f67 0x735311a 0x7353252 0x73532cd 0x7353352 0x73534ad 0x741abe3 0x747afff 0x7281372 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:16.098 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b364 /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xb6a317 0x1091994 0xf2eca0 0x9c6f140 0x9c6f43d 0x9c6f6b0 0x9c6f966 0x9c70415 0x9c70913 0x9c70f43 0x9c710ad 0x9d5141d 0x9d64220 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f3661e 0x6f38986 0x747af76 0x7299edd 0x7299fdf 0x729a0d7 0x7352eae 0x7352f67 0x735311a 0x7353252 0x73532cd 0x7353352 0x73534ad 0x741abe3 0x747afff 0x7281372 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:16.180 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x29fd4ba 0x2a522b0 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9f98c1c 0x9c16821 0x9c1705c 0xecfd0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47b04 2026-02-21T14:58:16.195 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a5230f 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:16.196 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a521d1 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:16.196 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a51607 0x7d8f27d 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:18.426 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:18.426 INFO:teuthology.orchestra.run.trial130.stderr:Suppressions used: 2026-02-21T14:58:18.427 INFO:teuthology.orchestra.run.trial130.stderr: count bytes template 2026-02-21T14:58:18.427 INFO:teuthology.orchestra.run.trial130.stderr: 1 8 MallocExtension::Register 2026-02-21T14:58:18.427 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:18.427 INFO:teuthology.orchestra.run.trial130.stderr: 2026-02-21T14:58:18.462 DEBUG:teuthology.orchestra.run.trial130:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-21T14:58:18.488 DEBUG:teuthology.orchestra.run.trial130:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T14:58:18.640 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T14:58:18.693 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:18,703 seastar - Reactor backend: linux-aio 2026-02-21T14:58:18.703 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:18,713 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:18.704 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:18,713 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:18.706 INFO:teuthology.orchestra.run.trial130.stderr:==39129==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T14:58:18.761 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:18,770 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-21T14:58:18.762 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:18,770 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-21T14:58:18.762 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:18,770 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-21T14:58:18.762 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:18,770 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-21T14:58:18.762 INFO:teuthology.orchestra.run.trial130.stderr:DEBUG 2026-02-21 14:58:18,771 [shard 0:main] seastar - reactor::drain 2026-02-21T14:58:20.800 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:20.800 INFO:teuthology.orchestra.run.trial130.stderr:Suppressions used: 2026-02-21T14:58:20.800 INFO:teuthology.orchestra.run.trial130.stderr: count bytes template 2026-02-21T14:58:20.800 INFO:teuthology.orchestra.run.trial130.stderr: 1 8 MallocExtension::Register 2026-02-21T14:58:20.800 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:20.800 INFO:teuthology.orchestra.run.trial130.stderr: 2026-02-21T14:58:20.820 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T14:58:20.820 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T14:58:20.862 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T14:58:20.862 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T14:58:20.869 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:20,878 seastar - Reactor backend: linux-aio 2026-02-21T14:58:20.880 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:20,889 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:20.880 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:20,889 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:20.884 INFO:teuthology.orchestra.run.trial130.stderr:==39127==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T14:58:20.885 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:20,894 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T14:58:20.944 INFO:teuthology.orchestra.run.trial130.stderr:DEBUG 2026-02-21 14:58:20,953 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-21T14:58:21.024 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x10919fe 0xf2eca0 0x9c6ef73 0x9c6f43d 0x9c6f6b0 0x9c713d7 0x9d505a2 0x9d50d1b 0x9d51206 0x9d51599 0x9d64220 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f927ce 0x6f965cf 0x6f9c179 0x74102d6 0x743293e 0x7282937 0x7282a2f 0x7353ce2 0x7353d9b 0x7353f52 0x74329bf 0x7282aac 0x7282b31 0x72ae77b 0x741b166 0x741b1eb 0x741b2c9 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.024 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x9d26457 0x9d67be6 0x789f8df 0x7bb0eff 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f927ce 0x6f965cf 0x6f9c179 0x74102d6 0x743293e 0x7282937 0x7282a2f 0x7353ce2 0x7353d9b 0x7353f52 0x74329bf 0x7282aac 0x7282b31 0x72ae77b 0x741b166 0x741b1eb 0x741b2c9 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.024 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x7483e95 0x73ca126 0x714c6d4 0x7bb0c56 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f927ce 0x6f965cf 0x6f9c179 0x74102d6 0x743293e 0x7282937 0x7282a2f 0x7353ce2 0x7353d9b 0x7353f52 0x74329bf 0x7282aac 0x7282b31 0x72ae77b 0x741b166 0x741b1eb 0x741b2c9 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.024 INFO:teuthology.orchestra.run.trial130.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff9bc1f7db 0xffffffff9bc1fb20 0xffffffff9ba9525e 0xffffffff9bde61a9 0xffffffff9bdc9715 0xffffffff9bdd3aeb 0xffffffff9bdd3f4b 0xffffffff9ba910d2 0xffffffff9c74d451 0xffffffff9c800ba2 2026-02-21T14:58:21.098 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x157eb 0x1091a0b 0xf2eca0 0x9c7399a 0x9c742d9 0x9d602a9 0x9d6304a 0x9d65ca7 0x9d26275 0x9d25275 0x7e6e01b 0x7e70e16 0x8f710e9 0x8f73a57 0x787509d 0x78c1e88 0x6f93999 0x754ac4a 0x754ad9e 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.099 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 66 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c62 /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xcbc464 0x9c0e63a 0x9d446da 0x9d60fbf 0x9d65ca7 0x9d26275 0x9d25275 0x7ba0164 0x7bb5485 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f935ba 0x754ac4a 0x754ad9e 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.099 INFO:teuthology.orchestra.run.trial130.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff9c127ef7 0xffffffff9bdf7f16 0xffffffff9bdfb8d0 0xffffffff9bdfcc31 0xffffffff9be24893 0xffffffff9be24a55 0xffffffff9bdcb80e 0xffffffff9bdd3aeb 0xffffffff9bdd3f4b 0xffffffff9ba910d2 0xffffffff9c74d451 0xffffffff9c800ba2 2026-02-21T14:58:21.121 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a51c74 0x7d8f28a 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9f98c1c 0x9c16821 0x9c1705c 0xecfd0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47b04 2026-02-21T14:58:21.163 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a5230f 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.163 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a5229e 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.166 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297b3 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a5230f 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.294 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15ef5 0x9c7851a 0x9c78aaf 0x9d615fc 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f3661e 0x6f38986 0x747af76 0x7299edd 0x7299fdf 0x729a0d7 0x7352eae 0x7352f67 0x735311a 0x7353252 0x73532cd 0x7353352 0x73534ad 0x741abe3 0x747afff 0x7281372 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.294 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x10917f7 0xf2eca0 0x9c6f140 0x9c6f43d 0x9c6f6b0 0x9c6f966 0x9c6ff28 0x9c783cf 0x9c78aaf 0x9d615fc 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f3661e 0x6f38986 0x747af76 0x7299edd 0x7299fdf 0x729a0d7 0x7352eae 0x7352f67 0x735311a 0x7353252 0x73532cd 0x7353352 0x73534ad 0x741abe3 0x747afff 0x7281372 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.294 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x10919fe 0xf2eca0 0x9d52025 0x9d5236a 0x9d526ed 0x9d62844 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f3661e 0x6f38986 0x747af76 0x7299edd 0x7299fdf 0x729a0d7 0x7352eae 0x7352f67 0x735311a 0x7353252 0x73532cd 0x7353352 0x73534ad 0x741abe3 0x747afff 0x7281372 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.377 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a5230f 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9f98c1c 0x9c16821 0x9c1705c 0xecfd0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47b04 2026-02-21T14:58:21.391 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x7d8f257 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.392 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x12985d /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a5230f 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:21.392 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x29fd48f 0x2a522b0 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:23.558 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:23.559 INFO:teuthology.orchestra.run.trial130.stderr:Suppressions used: 2026-02-21T14:58:23.559 INFO:teuthology.orchestra.run.trial130.stderr: count bytes template 2026-02-21T14:58:23.559 INFO:teuthology.orchestra.run.trial130.stderr: 1 8 MallocExtension::Register 2026-02-21T14:58:23.559 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:23.559 INFO:teuthology.orchestra.run.trial130.stderr: 2026-02-21T14:58:23.594 DEBUG:teuthology.orchestra.run.trial130:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-21T14:58:23.619 DEBUG:teuthology.orchestra.run.trial130:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T14:58:23.772 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T14:58:23.825 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:23,834 seastar - Reactor backend: linux-aio 2026-02-21T14:58:23.836 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:23,845 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:23.836 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:23,846 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:23.839 INFO:teuthology.orchestra.run.trial130.stderr:==39196==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T14:58:23.871 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x3476f35 0x34e5ab2 0x34e5dcb 0x34e6a73 0x34e6e76 0x34e71c1 0x3583884 0x3583d41 0x35840b6 0x35c9ded 0x35ca5a7 0x35ca8a9 0x35caa3a 0x35cace3 0x35cafea 0x35cb130 0x357b046 0x358162e 0x3581874 0x35819c8 0x3581a76 0x3581d40 0x3581f43 0x35821c4 0x3582386 0x35824a5 0x357b046 0x357b3f2 0x357b638 0x357b70d 0x357d64e 0x357dca7 0x357dedb 0x357e0d6 0x357e469 0x358e231 0x358e6e7 0x358eb24 0x358ee4a 0x358ef6c 0x359142e 0x35932c7 0x359357e 0x359460f 0x35946cd 0x359476f 0x3594898 0x3594bbd 0x3594dbf 0x3595141 0x35cc550 0x35ccdfa 0x35cd35a 0x35cd6d2 0x35cdb70 0x35cdfaf 0x3595a90 0x3596ac7 0x3596d7e 0x359721d 0x3597339 0x35975ed 0x359797f 0x3597cb2 0x359800f 0x3598247 0x3598882 0x3598bb2 0x3598f18 0x35993c5 0x3599804 0x3599a30 0x3599e3f 0x359a0f6 0x359a247 0x359a518 0x359a8a3 0x359abd6 0x359b0c9 0x359b4ab 0x359b637 0x359ba6f 0x359bddb 0x359c140 0x359c463 0x359c545 0x359c7d3 0x359c9aa 0x359cc11 0x35cffc9 0x35d0719 2026-02-21T14:58:23.894 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:23,903 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-21T14:58:23.894 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:23,903 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-21T14:58:23.895 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:23,903 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-21T14:58:23.895 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:23,903 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-21T14:58:23.895 INFO:teuthology.orchestra.run.trial130.stderr:DEBUG 2026-02-21 14:58:23,903 [shard 0:main] seastar - reactor::drain 2026-02-21T14:58:25.960 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:25.960 INFO:teuthology.orchestra.run.trial130.stderr:Suppressions used: 2026-02-21T14:58:25.960 INFO:teuthology.orchestra.run.trial130.stderr: count bytes template 2026-02-21T14:58:25.960 INFO:teuthology.orchestra.run.trial130.stderr: 1 8 MallocExtension::Register 2026-02-21T14:58:25.960 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:25.960 INFO:teuthology.orchestra.run.trial130.stderr: 2026-02-21T14:58:25.980 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T14:58:25.980 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T14:58:26.021 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T14:58:26.021 INFO:teuthology.orchestra.run.trial130.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T14:58:26.028 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:26,037 seastar - Reactor backend: linux-aio 2026-02-21T14:58:26.040 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:26,049 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:26.040 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:26,049 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T14:58:26.044 INFO:teuthology.orchestra.run.trial130.stderr:==39194==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T14:58:26.044 INFO:teuthology.orchestra.run.trial130.stderr:INFO 2026-02-21 14:58:26,053 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T14:58:26.101 INFO:teuthology.orchestra.run.trial130.stderr:DEBUG 2026-02-21 14:58:26,110 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-21T14:58:26.181 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x361f /lib64/libasan.so.8+0x157c7 0x1091a0b 0xf2eca0 0x9c6ef73 0x9c6f43d 0x9c6f6b0 0x9c713d7 0x9d54dae 0x9d56a00 0x9d5705a 0x9d575a0 0x9d6182e 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f927ce 0x6f965cf 0x6f9c179 0x74102d6 0x743293e 0x7282937 0x7282a2f 0x7353ce2 0x7353d9b 0x7353f52 0x74329bf 0x7282aac 0x7282b31 0x72ae77b 0x741b166 0x741b1eb 0x741b2c9 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:26.181 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x10919fc 0xf2eca0 0x9c7399a 0x9c73c47 0x9d61139 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f927ce 0x6f965cf 0x6f9c179 0x74102d6 0x743293e 0x7282937 0x7282a2f 0x7353ce2 0x7353d9b 0x7353f52 0x74329bf 0x7282aac 0x7282b31 0x72ae77b 0x741b166 0x741b1eb 0x741b2c9 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:26.181 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9b439fb 0x9bc1196 0x9bc12ac 0x9bc1388 0x9bc1a44 0x9bc16bd 0x9bc1bab 0x9bc1faf 0x9bc226c 0x9d63e75 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f927ce 0x6f965cf 0x6f9c179 0x74102d6 0x743293e 0x7282937 0x7282a2f 0x7353ce2 0x7353d9b 0x7353f52 0x74329bf 0x7282aac 0x7282b31 0x72ae77b 0x741b166 0x741b1eb 0x741b2c9 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:26.181 INFO:teuthology.orchestra.run.trial130.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff9ba9519a 0xffffffff9bde61a9 0xffffffff9bdc9715 0xffffffff9bdd3aeb 0xffffffff9bdd3f4b 0xffffffff9ba910d2 0xffffffff9c74d451 0xffffffff9c800ba2 2026-02-21T14:58:26.216 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 68 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x9c712f3 0x9c71924 0x9c71b23 0x9d56e45 0x9d575a0 0x9d6182e 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f935ba 0x754ac4a 0x754ad9e 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9f98c1c 0x9c16821 0x9c1705c 0xecfd0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47b04 2026-02-21T14:58:26.255 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a51298 0x7d8f268 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9f98c1c 0x9c16821 0x9c1705c 0xecfd0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47b04 2026-02-21T14:58:26.319 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a5166d 0x7d8f27d 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:26.320 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x2a5230f 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:26.320 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a51c16 0x7d8f28a 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:26.445 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x9c6f39a 0x9c6f6b0 0x9c713d7 0x9c71924 0x9c71b23 0x9d56e45 0x9d575a0 0x9d6182e 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f3661e 0x6f38986 0x747af76 0x7299edd 0x7299fdf 0x729a0d7 0x7352eae 0x7352f67 0x735311a 0x7353252 0x73532cd 0x7353352 0x73534ad 0x741abe3 0x747afff 0x7281372 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:26.445 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfdf27 /lib64/libasan.so.8+0xeb6b /lib64/libasan.so.8+0xef905 0x316e828 0x316ea02 0x316ecb9 0x316ec7e 0x316edae 0x6f6256e 0x9d468b2 0x9d46aca 0x9d46d9c 0x7bb160b 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f3661e 0x6f38986 0x747af76 0x7299edd 0x7299fdf 0x729a0d7 0x7352eae 0x7352f67 0x735311a 0x7353252 0x73532cd 0x7353352 0x73534ad 0x741abe3 0x747afff 0x7281372 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:26.445 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0xe0758f 0x11264d9 0xf2ec95 0x9d52025 0x9d5236a 0x9d526ed 0x9d62844 0x9d65ca7 0x9d26275 0x9d25275 0x7bb15cd 0x7bbb096 0x7a3b74b 0x7a7bab3 0x7111f37 0x6f3661e 0x6f38986 0x747af76 0x7299edd 0x7299fdf 0x729a0d7 0x7352eae 0x7352f67 0x735311a 0x7353252 0x73532cd 0x7353352 0x73534ad 0x741abe3 0x747afff 0x7281372 0x729a254 0x73a7d88 0x74274fc 0x7427577 0x74275f2 0x74eb1a2 0x75d925c 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:26.502 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x2a5230f 0x7d8f24f 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9f98c1c 0x9c16821 0x9c1705c 0xecfd0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47b04 2026-02-21T14:58:26.542 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a51cd9 0x7d8f28a 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:26.542 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a511e1 0x7d8f268 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:26.543 INFO:teuthology.orchestra.run.trial130.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e4665c 0x9e46945 0x9ea72e3 0x9ea7ad6 0x9ea7d44 0x9ea7e8c 0x9ea834b /lib64/libc.so.6+0x3fc2f 0x2a5165c 0x7d8f27d 0x7dab998 0x7e4d6ff 0x7dcbaf9 0x787047b 0x7871170 0x79a22f0 0x7831428 0x78c04a4 0x78fbe21 0x78fc09a 0x79913af 0x799150e 0x79a2453 0x785fa5c 0x7981757 0x79817e6 0x798188a 0x73dd41e 0x73fd4c1 0x7400c44 0x753dfb0 0x9f011ab 0x9e55480 0x9f2ec1b 0x9f973fe 0x9fb1e8e 0x9fb250a 0x26ec7eb 0xa5c0410 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T14:58:28.698 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:28.699 INFO:teuthology.orchestra.run.trial130.stderr:Suppressions used: 2026-02-21T14:58:28.699 INFO:teuthology.orchestra.run.trial130.stderr: count bytes template 2026-02-21T14:58:28.699 INFO:teuthology.orchestra.run.trial130.stderr: 1 8 MallocExtension::Register 2026-02-21T14:58:28.699 INFO:teuthology.orchestra.run.trial130.stderr:----------------------------------------------------- 2026-02-21T14:58:28.699 INFO:teuthology.orchestra.run.trial130.stderr: 2026-02-21T14:58:28.735 DEBUG:teuthology.orchestra.run.trial130:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-21T14:58:28.760 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-21T14:58:28.761 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:28.761 DEBUG:teuthology.orchestra.run.trial130:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-21T14:58:28.825 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:28.826 DEBUG:teuthology.orchestra.run.trial130:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-21T14:58:28.889 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:28.889 DEBUG:teuthology.orchestra.run.trial130:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-21T14:58:28.954 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:28.954 DEBUG:teuthology.orchestra.run.trial130:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-21T14:58:29.018 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:29.018 DEBUG:teuthology.orchestra.run.trial130:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-21T14:58:29.074 INFO:tasks.ceph:Adding keys to all mons... 2026-02-21T14:58:29.074 DEBUG:teuthology.orchestra.run.trial130:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-21T14:58:29.137 INFO:teuthology.orchestra.run.trial130.stdout:[mgr.x] 2026-02-21T14:58:29.138 INFO:teuthology.orchestra.run.trial130.stdout: key = AQACyJlppJZVJhAAxLQQlo6jwKmrb/xFqpecBg== 2026-02-21T14:58:29.138 INFO:teuthology.orchestra.run.trial130.stdout:[osd.0] 2026-02-21T14:58:29.138 INFO:teuthology.orchestra.run.trial130.stdout: key = AQAHyJlpWd28LRAANKzOTYgmYfzhnyfj5WE37Q== 2026-02-21T14:58:29.138 INFO:teuthology.orchestra.run.trial130.stdout:[osd.1] 2026-02-21T14:58:29.138 INFO:teuthology.orchestra.run.trial130.stdout: key = AQAMyJlpknoPORAAUDgRZnbDPaMC1Vdi9fo2XQ== 2026-02-21T14:58:29.138 INFO:teuthology.orchestra.run.trial130.stdout:[osd.2] 2026-02-21T14:58:29.138 INFO:teuthology.orchestra.run.trial130.stdout: key = AQASyJlp/zPLBhAAi9ZUNIJrtSxfJIles1cjfg== 2026-02-21T14:58:29.138 INFO:teuthology.orchestra.run.trial130.stdout:[client.0] 2026-02-21T14:58:29.138 INFO:teuthology.orchestra.run.trial130.stdout: key = AQACyJlpGin4KBAAUu3+klFuB7HOmNQkTjMkqQ== 2026-02-21T14:58:29.140 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-21T14:58:29.228 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T14:58:29.276 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T14:58:29.324 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T14:58:29.374 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-21T14:58:29.424 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-21T14:58:29.424 DEBUG:teuthology.orchestra.run.trial130:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-21T14:58:29.449 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T14:58:29.534 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:29.542+0000 7f6101c2cc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.534 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:29.542+0000 7f6101c2cc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.546 DEBUG:teuthology.orchestra.run.trial130:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-21T14:58:29.571 DEBUG:teuthology.orchestra.run.trial130:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-21T14:58:29.627 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-21T14:58:29.628 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-21T14:58:29.628 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-21T14:58:29.670 INFO:tasks.ceph.mon.a:Started 2026-02-21T14:58:29.670 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-21T14:58:29.671 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-21T14:58:29.671 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-21T14:58:29.673 INFO:tasks.ceph.mgr.x:Started 2026-02-21T14:58:29.673 DEBUG:tasks.ceph:set 0 configs 2026-02-21T14:58:29.673 DEBUG:teuthology.orchestra.run.trial130:> sudo ceph --cluster ceph config dump 2026-02-21T14:58:29.733 INFO:tasks.ceph.mon.a.trial130.stderr:2026-02-21T14:58:29.741+0000 7f7d47253c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.734 INFO:tasks.ceph.mon.a.trial130.stderr:2026-02-21T14:58:29.741+0000 7f7d47253c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.734 INFO:tasks.ceph.mon.a.trial130.stderr:2026-02-21T14:58:29.741+0000 7f7d47253c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.737 INFO:tasks.ceph.mgr.x.trial130.stderr:2026-02-21T14:58:29.745+0000 7fbebe3cbfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.747 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:29.754+0000 7f7e421e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.752 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:29.760+0000 7f7e421e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.786 INFO:teuthology.orchestra.run.trial130.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-21T14:58:29.795 INFO:tasks.ceph:Setting crush tunables to default 2026-02-21T14:58:29.795 DEBUG:teuthology.orchestra.run.trial130:> sudo ceph --cluster ceph osd crush tunables default 2026-02-21T14:58:29.869 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:29.874+0000 7fdc8bf0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.872 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:29.880+0000 7fdc8bf0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.907 INFO:teuthology.orchestra.run.trial130.stderr:adjusted tunables profile to default 2026-02-21T14:58:29.915 INFO:tasks.ceph:check_enable_crimson: True 2026-02-21T14:58:29.915 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-21T14:58:29.915 DEBUG:teuthology.orchestra.run.trial130:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-21T14:58:29.941 INFO:tasks.ceph.mgr.x.trial130.stderr:2026-02-21T14:58:29.949+0000 7fbebe3cbfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.941 INFO:tasks.ceph.mgr.x.trial130.stderr:2026-02-21T14:58:29.950+0000 7fbebe3cbfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.990 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:29.995+0000 7f14a4a8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:29.992 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:30.000+0000 7f14a4a8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:30.032 INFO:tasks.ceph.mon.a.trial130.stderr:2026-02-21T14:58:30.040+0000 7f7d3e5f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-21T14:58:30.032 INFO:tasks.ceph.mon.a.trial130.stderr:Please be aware that this feature is experimental, untested, 2026-02-21T14:58:30.032 INFO:tasks.ceph.mon.a.trial130.stderr:unsupported, and may result in data corruption, data loss, 2026-02-21T14:58:30.032 INFO:tasks.ceph.mon.a.trial130.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-21T14:58:30.032 INFO:tasks.ceph.mon.a.trial130.stderr:feature with important data. 2026-02-21T14:58:30.032 INFO:tasks.ceph.mon.a.trial130.stderr: 2026-02-21T14:58:30.773 INFO:tasks.ceph.mgr.x.trial130.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-21T14:58:30.773 INFO:tasks.ceph.mgr.x.trial130.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-21T14:58:30.773 INFO:tasks.ceph.mgr.x.trial130.stderr: from numpy import show_config as show_numpy_config 2026-02-21T14:58:30.918 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-21T14:58:30.918 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:30.918 DEBUG:teuthology.orchestra.run.trial130:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-21T14:58:30.983 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:30.983 DEBUG:teuthology.orchestra.run.trial130:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-21T14:58:31.047 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:58:31.047 DEBUG:teuthology.orchestra.run.trial130:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-21T14:58:31.111 DEBUG:teuthology.orchestra.run.trial130:> sudo ceph --cluster ceph osd new 19de4c87-86a9-4917-8572-358cf6c28af1 0 2026-02-21T14:58:31.225 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:31.230+0000 7f8d1f4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:31.228 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:31.236+0000 7f8d1f4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:31.266 INFO:teuthology.orchestra.run.trial130.stdout:0 2026-02-21T14:58:31.275 DEBUG:teuthology.orchestra.run.trial130:> sudo ceph --cluster ceph osd new be1c4136-6a34-402e-88fe-5c6867ae35ac 1 2026-02-21T14:58:31.347 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:31.352+0000 7f8550d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:31.350 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:31.358+0000 7f8550d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:31.387 INFO:teuthology.orchestra.run.trial130.stdout:1 2026-02-21T14:58:31.397 DEBUG:teuthology.orchestra.run.trial130:> sudo ceph --cluster ceph osd new 6df6fcfe-eebc-4350-a5d3-9d885e6a904a 2 2026-02-21T14:58:31.471 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:31.476+0000 7f463b85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:31.473 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:31.482+0000 7f463b85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:31.511 INFO:teuthology.orchestra.run.trial130.stdout:2 2026-02-21T14:58:31.521 DEBUG:teuthology.orchestra.run.trial130:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-21T14:58:31.548 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-21T14:58:31.549 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-21T14:58:31.591 INFO:tasks.ceph.osd.0:Started 2026-02-21T14:58:31.591 DEBUG:teuthology.orchestra.run.trial130:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-21T14:58:31.618 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-21T14:58:31.618 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-21T14:58:31.662 INFO:tasks.ceph.osd.1:Started 2026-02-21T14:58:31.662 DEBUG:teuthology.orchestra.run.trial130:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-21T14:58:31.691 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-21T14:58:31.691 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-21T14:58:31.732 INFO:tasks.ceph.osd.2:Started 2026-02-21T14:58:31.733 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T14:58:31.810 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:31.816+0000 7f519ca5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:31.812 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:31.821+0000 7f519ca5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:31.846 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:31.846 INFO:teuthology.orchestra.run.trial130.stdout:{"epoch":6,"fsid":"05ac4a4e-7e84-4852-9cff-af5d19cabadf","created":"2026-02-21T14:58:29.754604+0000","modified":"2026-02-21T14:58:31.519727+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T14:58:31.519727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"19de4c87-86a9-4917-8572-358cf6c28af1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"be1c4136-6a34-402e-88fe-5c6867ae35ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6df6fcfe-eebc-4350-a5d3-9d885e6a904a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T14:58:31.855 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-21T14:58:31.855 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-21T14:58:32.157 DEBUG:teuthology.orchestra.run.trial130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T14:58:32.227 INFO:teuthology.misc.health.trial130.stderr:2026-02-21T14:58:32.233+0000 7fad5ef9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:32.230 INFO:teuthology.misc.health.trial130.stderr:2026-02-21T14:58:32.238+0000 7fad5ef9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:32.263 INFO:teuthology.misc.health.trial130.stdout: 2026-02-21T14:58:32.263 INFO:teuthology.misc.health.trial130.stdout:{"epoch":6,"fsid":"05ac4a4e-7e84-4852-9cff-af5d19cabadf","created":"2026-02-21T14:58:29.754604+0000","modified":"2026-02-21T14:58:31.519727+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T14:58:31.519727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"19de4c87-86a9-4917-8572-358cf6c28af1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"be1c4136-6a34-402e-88fe-5c6867ae35ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6df6fcfe-eebc-4350-a5d3-9d885e6a904a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T14:58:32.272 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-21T14:58:33.163 INFO:tasks.ceph.mgr.x.trial130.stderr:2026-02-21T14:58:33.171+0000 7fbdd9283640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-21T14:58:38.574 DEBUG:teuthology.orchestra.run.trial130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T14:58:38.659 INFO:teuthology.misc.health.trial130.stderr:2026-02-21T14:58:38.664+0000 7f5985006640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:38.663 INFO:teuthology.misc.health.trial130.stderr:2026-02-21T14:58:38.671+0000 7f5985006640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:38.802 INFO:teuthology.misc.health.trial130.stdout: 2026-02-21T14:58:38.802 INFO:teuthology.misc.health.trial130.stdout:{"epoch":7,"fsid":"05ac4a4e-7e84-4852-9cff-af5d19cabadf","created":"2026-02-21T14:58:29.754604+0000","modified":"2026-02-21T14:58:38.027372+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T14:58:31.519727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"19de4c87-86a9-4917-8572-358cf6c28af1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"be1c4136-6a34-402e-88fe-5c6867ae35ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6df6fcfe-eebc-4350-a5d3-9d885e6a904a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T14:58:38.817 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-21T14:58:45.120 DEBUG:teuthology.orchestra.run.trial130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T14:58:45.190 INFO:teuthology.misc.health.trial130.stderr:2026-02-21T14:58:45.196+0000 7feb9405a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:45.192 INFO:teuthology.misc.health.trial130.stderr:2026-02-21T14:58:45.200+0000 7feb9405a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:45.305 INFO:teuthology.misc.health.trial130.stdout: 2026-02-21T14:58:45.305 INFO:teuthology.misc.health.trial130.stdout:{"epoch":13,"fsid":"05ac4a4e-7e84-4852-9cff-af5d19cabadf","created":"2026-02-21T14:58:29.754604+0000","modified":"2026-02-21T14:58:45.050517+0000","last_up_change":"2026-02-21T14:58:41.036030+0000","last_in_change":"2026-02-21T14:58:31.519727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T14:58:42.184030+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19de4c87-86a9-4917-8572-358cf6c28af1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6805","nonce":1758314576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6804","nonce":1758314576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6808","nonce":1758314576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6809","nonce":1758314576}]},"public_addr":"10.20.193.130:6805/1758314576","cluster_addr":"10.20.193.130:6804/1758314576","heartbeat_back_addr":"10.20.193.130:6808/1758314576","heartbeat_front_addr":"10.20.193.130:6809/1758314576","state":["exists","up"]},{"osd":1,"uuid":"be1c4136-6a34-402e-88fe-5c6867ae35ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6811","nonce":1995513605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6810","nonce":1995513605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6806","nonce":1995513605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6807","nonce":1995513605}]},"public_addr":"10.20.193.130:6811/1995513605","cluster_addr":"10.20.193.130:6810/1995513605","heartbeat_back_addr":"10.20.193.130:6806/1995513605","heartbeat_front_addr":"10.20.193.130:6807/1995513605","state":["exists","up"]},{"osd":2,"uuid":"6df6fcfe-eebc-4350-a5d3-9d885e6a904a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6803","nonce":2889879952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6802","nonce":2889879952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6817","nonce":2889879952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6816","nonce":2889879952}]},"public_addr":"10.20.193.130:6803/2889879952","cluster_addr":"10.20.193.130:6802/2889879952","heartbeat_back_addr":"10.20.193.130:6817/2889879952","heartbeat_front_addr":"10.20.193.130:6816/2889879952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T14:58:45.314 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-21T14:58:45.314 INFO:tasks.ceph:Creating RBD pool 2026-02-21T14:58:45.314 DEBUG:teuthology.orchestra.run.trial130:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-21T14:58:45.389 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:45.393+0000 7f2a104cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:45.391 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:45.399+0000 7f2a104cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:46.046 INFO:teuthology.orchestra.run.trial130.stderr:pool 'rbd' created 2026-02-21T14:58:46.058 DEBUG:teuthology.orchestra.run.trial130:> rbd --cluster ceph pool init rbd 2026-02-21T14:58:46.102 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:46.110+0000 7fd0f1078240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:46.102 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:46.110+0000 7fd0f1078240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:46.105 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:46.113+0000 7fd0f1078240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:49.104 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-21T14:58:49.105 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-21T14:58:49.105 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-21T14:58:49.222 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:49.227+0000 7f66d5fb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:49.224 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:49.232+0000 7f66d5fb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:49.338 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:49.347 INFO:teuthology.orchestra.run.trial130.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-21T14:58:49.348 INFO:tasks.ceph_manager:config epoch is 1 2026-02-21T14:58:49.348 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-21T14:58:49.348 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-21T14:58:49.348 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-21T14:58:49.424 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:49.431+0000 7ff136b8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:49.427 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:49.435+0000 7ff136b8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:49.575 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:49.583 INFO:teuthology.orchestra.run.trial130.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6800","nonce":2871005989},{"type":"v1","addr":"10.20.193.130:6801","nonce":2871005989}]},"active_addr":"10.20.193.130:6801/2871005989","active_change":"2026-02-21T14:58:32.166250+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.130:0","nonce":2828990678}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.130:0","nonce":1431861095}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.130:0","nonce":1842820374}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.130:0","nonce":2076976944}]}]} 2026-02-21T14:58:49.587 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-21T14:58:49.587 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-21T14:58:49.588 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T14:58:49.666 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:49.671+0000 7f90d9f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:49.668 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:49.676+0000 7f90d9f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:49.784 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:49.784 INFO:teuthology.orchestra.run.trial130.stdout:{"epoch":17,"fsid":"05ac4a4e-7e84-4852-9cff-af5d19cabadf","created":"2026-02-21T14:58:29.754604+0000","modified":"2026-02-21T14:58:49.068382+0000","last_up_change":"2026-02-21T14:58:41.036030+0000","last_in_change":"2026-02-21T14:58:31.519727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T14:58:42.184030+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T14:58:45.519400+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19de4c87-86a9-4917-8572-358cf6c28af1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6805","nonce":1758314576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6804","nonce":1758314576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6808","nonce":1758314576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6809","nonce":1758314576}]},"public_addr":"10.20.193.130:6805/1758314576","cluster_addr":"10.20.193.130:6804/1758314576","heartbeat_back_addr":"10.20.193.130:6808/1758314576","heartbeat_front_addr":"10.20.193.130:6809/1758314576","state":["exists","up"]},{"osd":1,"uuid":"be1c4136-6a34-402e-88fe-5c6867ae35ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6811","nonce":1995513605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6810","nonce":1995513605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6806","nonce":1995513605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6807","nonce":1995513605}]},"public_addr":"10.20.193.130:6811/1995513605","cluster_addr":"10.20.193.130:6810/1995513605","heartbeat_back_addr":"10.20.193.130:6806/1995513605","heartbeat_front_addr":"10.20.193.130:6807/1995513605","state":["exists","up"]},{"osd":2,"uuid":"6df6fcfe-eebc-4350-a5d3-9d885e6a904a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6803","nonce":2889879952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6802","nonce":2889879952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6817","nonce":2889879952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6816","nonce":2889879952}]},"public_addr":"10.20.193.130:6803/2889879952","cluster_addr":"10.20.193.130:6802/2889879952","heartbeat_back_addr":"10.20.193.130:6817/2889879952","heartbeat_front_addr":"10.20.193.130:6816/2889879952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T14:58:49.794 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-21T14:58:49.794 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T14:58:49.871 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:49.877+0000 7f9e02446640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:49.873 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:49.881+0000 7f9e02446640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:49.986 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:49.987 INFO:teuthology.orchestra.run.trial130.stdout:{"epoch":17,"fsid":"05ac4a4e-7e84-4852-9cff-af5d19cabadf","created":"2026-02-21T14:58:29.754604+0000","modified":"2026-02-21T14:58:49.068382+0000","last_up_change":"2026-02-21T14:58:41.036030+0000","last_in_change":"2026-02-21T14:58:31.519727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T14:58:42.184030+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T14:58:45.519400+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19de4c87-86a9-4917-8572-358cf6c28af1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6805","nonce":1758314576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6804","nonce":1758314576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6808","nonce":1758314576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6809","nonce":1758314576}]},"public_addr":"10.20.193.130:6805/1758314576","cluster_addr":"10.20.193.130:6804/1758314576","heartbeat_back_addr":"10.20.193.130:6808/1758314576","heartbeat_front_addr":"10.20.193.130:6809/1758314576","state":["exists","up"]},{"osd":1,"uuid":"be1c4136-6a34-402e-88fe-5c6867ae35ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6811","nonce":1995513605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6810","nonce":1995513605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6806","nonce":1995513605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6807","nonce":1995513605}]},"public_addr":"10.20.193.130:6811/1995513605","cluster_addr":"10.20.193.130:6810/1995513605","heartbeat_back_addr":"10.20.193.130:6806/1995513605","heartbeat_front_addr":"10.20.193.130:6807/1995513605","state":["exists","up"]},{"osd":2,"uuid":"6df6fcfe-eebc-4350-a5d3-9d885e6a904a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6803","nonce":2889879952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6802","nonce":2889879952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6817","nonce":2889879952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6816","nonce":2889879952}]},"public_addr":"10.20.193.130:6803/2889879952","cluster_addr":"10.20.193.130:6802/2889879952","heartbeat_back_addr":"10.20.193.130:6817/2889879952","heartbeat_front_addr":"10.20.193.130:6816/2889879952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T14:58:49.997 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T14:58:49.997 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T14:58:49.998 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-21T14:58:50.080 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.085+0000 7fd4e8acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.080 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.085+0000 7f3a567a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.081 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.086+0000 7f448965a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.082 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.090+0000 7fd4e8acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.082 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.090+0000 7f3a567a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.083 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.091+0000 7f448965a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.100 INFO:teuthology.orchestra.run.trial130.stdout:38654705674 2026-02-21T14:58:50.101 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T14:58:50.102 INFO:teuthology.orchestra.run.trial130.stdout:42949672969 2026-02-21T14:58:50.102 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T14:58:50.105 INFO:teuthology.orchestra.run.trial130.stdout:42949672969 2026-02-21T14:58:50.105 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T14:58:50.219 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.225+0000 7f83cabc1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.222 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.230+0000 7f83cabc1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.224 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.229+0000 7f848ad9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.230 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.238+0000 7f848ad9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.267 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.272+0000 7fb9cfbdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.269 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:50.278+0000 7fb9cfbdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:50.340 INFO:teuthology.orchestra.run.trial130.stdout:38654705669 2026-02-21T14:58:50.345 INFO:teuthology.orchestra.run.trial130.stdout:42949672964 2026-02-21T14:58:50.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705669 for osd.1 2026-02-21T14:58:50.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672964 for osd.2 2026-02-21T14:58:50.385 INFO:teuthology.orchestra.run.trial130.stdout:42949672964 2026-02-21T14:58:50.396 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672964 for osd.0 2026-02-21T14:58:51.352 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T14:58:51.357 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T14:58:51.397 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T14:58:51.430 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:51.435+0000 7f98b8e46640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:51.433 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:51.441+0000 7f98b8e46640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:51.473 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:51.478+0000 7f849ae8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:51.473 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:51.478+0000 7f1e4b10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:51.475 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:51.484+0000 7f849ae8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:51.476 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:51.484+0000 7f1e4b10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:51.548 INFO:teuthology.orchestra.run.trial130.stdout:38654705674 2026-02-21T14:58:51.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705674 for osd.1 2026-02-21T14:58:51.559 DEBUG:teuthology.parallel:result is None 2026-02-21T14:58:51.592 INFO:teuthology.orchestra.run.trial130.stdout:42949672969 2026-02-21T14:58:51.593 INFO:teuthology.orchestra.run.trial130.stdout:42949672969 2026-02-21T14:58:51.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.0 2026-02-21T14:58:51.605 DEBUG:teuthology.parallel:result is None 2026-02-21T14:58:51.605 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.2 2026-02-21T14:58:51.606 DEBUG:teuthology.parallel:result is None 2026-02-21T14:58:51.606 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T14:58:51.606 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T14:58:51.721 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:51.727+0000 7f9d880c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:51.723 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:51.732+0000 7f9d880c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:51.835 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:51.836 INFO:teuthology.orchestra.run.trial130.stderr:dumped all 2026-02-21T14:58:51.845 INFO:teuthology.orchestra.run.trial130.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2026-02-21T14:58:50.170718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2540,"kb_used_data":2540,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497428,"statfs":{"total":160255967232,"available":160253366272,"internally_reserved":0,"allocated":2600960,"data_stored":2600960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.108596"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.090430+0000","last_change":"2026-02-21T14:58:49.096029+0000","last_active":"2026-02-21T14:58:49.090430+0000","last_peered":"2026-02-21T14:58:49.090430+0000","last_clean":"2026-02-21T14:58:49.090430+0000","last_became_active":"2026-02-21T14:58:47.128560+0000","last_became_peered":"2026-02-21T14:58:47.128560+0000","last_unstale":"2026-02-21T14:58:49.090430+0000","last_undegraded":"2026-02-21T14:58:49.090430+0000","last_fullsized":"2026-02-21T14:58:49.090430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.091993+0000","last_change":"2026-02-21T14:58:49.095614+0000","last_active":"2026-02-21T14:58:49.091993+0000","last_peered":"2026-02-21T14:58:49.091993+0000","last_clean":"2026-02-21T14:58:49.091993+0000","last_became_active":"2026-02-21T14:58:47.125613+0000","last_became_peered":"2026-02-21T14:58:47.125613+0000","last_unstale":"2026-02-21T14:58:49.091993+0000","last_undegraded":"2026-02-21T14:58:49.091993+0000","last_fullsized":"2026-02-21T14:58:49.091993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.091965+0000","last_change":"2026-02-21T14:58:49.098251+0000","last_active":"2026-02-21T14:58:49.091965+0000","last_peered":"2026-02-21T14:58:49.091965+0000","last_clean":"2026-02-21T14:58:49.091965+0000","last_became_active":"2026-02-21T14:58:47.134964+0000","last_became_peered":"2026-02-21T14:58:47.134964+0000","last_unstale":"2026-02-21T14:58:49.091965+0000","last_undegraded":"2026-02-21T14:58:49.091965+0000","last_fullsized":"2026-02-21T14:58:49.091965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.092255+0000","last_change":"2026-02-21T14:58:49.097230+0000","last_active":"2026-02-21T14:58:49.092255+0000","last_peered":"2026-02-21T14:58:49.092255+0000","last_clean":"2026-02-21T14:58:49.092255+0000","last_became_active":"2026-02-21T14:58:47.138164+0000","last_became_peered":"2026-02-21T14:58:47.138164+0000","last_unstale":"2026-02-21T14:58:49.092255+0000","last_undegraded":"2026-02-21T14:58:49.092255+0000","last_fullsized":"2026-02-21T14:58:49.092255+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.097095+0000","last_change":"2026-02-21T14:58:49.098467+0000","last_active":"2026-02-21T14:58:49.097095+0000","last_peered":"2026-02-21T14:58:49.097095+0000","last_clean":"2026-02-21T14:58:49.097095+0000","last_became_active":"2026-02-21T14:58:47.153243+0000","last_became_peered":"2026-02-21T14:58:47.153243+0000","last_unstale":"2026-02-21T14:58:49.097095+0000","last_undegraded":"2026-02-21T14:58:49.097095+0000","last_fullsized":"2026-02-21T14:58:49.097095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.089931+0000","last_change":"2026-02-21T14:58:49.093347+0000","last_active":"2026-02-21T14:58:49.089931+0000","last_peered":"2026-02-21T14:58:49.089931+0000","last_clean":"2026-02-21T14:58:49.089931+0000","last_became_active":"2026-02-21T14:58:47.146202+0000","last_became_peered":"2026-02-21T14:58:47.146202+0000","last_unstale":"2026-02-21T14:58:49.089931+0000","last_undegraded":"2026-02-21T14:58:49.089931+0000","last_fullsized":"2026-02-21T14:58:49.089931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.088015+0000","last_change":"2026-02-21T14:58:49.092137+0000","last_active":"2026-02-21T14:58:49.088015+0000","last_peered":"2026-02-21T14:58:49.088015+0000","last_clean":"2026-02-21T14:58:49.088015+0000","last_became_active":"2026-02-21T14:58:47.160042+0000","last_became_peered":"2026-02-21T14:58:47.160042+0000","last_unstale":"2026-02-21T14:58:49.088015+0000","last_undegraded":"2026-02-21T14:58:49.088015+0000","last_fullsized":"2026-02-21T14:58:49.088015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.092216+0000","last_change":"2026-02-21T14:58:49.098217+0000","last_active":"2026-02-21T14:58:49.092216+0000","last_peered":"2026-02-21T14:58:49.092216+0000","last_clean":"2026-02-21T14:58:49.092216+0000","last_became_active":"2026-02-21T14:58:47.124462+0000","last_became_peered":"2026-02-21T14:58:47.124462+0000","last_unstale":"2026-02-21T14:58:49.092216+0000","last_undegraded":"2026-02-21T14:58:49.092216+0000","last_fullsized":"2026-02-21T14:58:49.092216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.089339+0000","last_change":"2026-02-21T14:58:44.102806+0000","last_active":"2026-02-21T14:58:49.089339+0000","last_peered":"2026-02-21T14:58:49.089339+0000","last_clean":"2026-02-21T14:58:49.089339+0000","last_became_active":"2026-02-21T14:58:44.090205+0000","last_became_peered":"2026-02-21T14:58:44.090205+0000","last_unstale":"2026-02-21T14:58:49.089339+0000","last_undegraded":"2026-02-21T14:58:49.089339+0000","last_fullsized":"2026-02-21T14:58:49.089339+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_clean_scrub_stamp":"2026-02-21T14:58:43.041461+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253366272,"internally_reserved":0,"allocated":2600960,"data_stored":2600960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":316,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166340,"statfs":{"total":53418655744,"available":53418332160,"internally_reserved":0,"allocated":323584,"data_stored":323584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1076,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165580,"statfs":{"total":53418655744,"available":53417553920,"internally_reserved":0,"allocated":1101824,"data_stored":1101824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1148,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165508,"statfs":{"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417553920,"internally_reserved":0,"allocated":1101824,"data_stored":1101824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418332160,"internally_reserved":0,"allocated":323584,"data_stored":323584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T14:58:51.846 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T14:58:51.922 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:51.928+0000 7f905250b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:51.924 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:51.932+0000 7f905250b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:52.038 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:52.038 INFO:teuthology.orchestra.run.trial130.stderr:dumped all 2026-02-21T14:58:52.048 INFO:teuthology.orchestra.run.trial130.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2026-02-21T14:58:50.170718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2540,"kb_used_data":2540,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497428,"statfs":{"total":160255967232,"available":160253366272,"internally_reserved":0,"allocated":2600960,"data_stored":2600960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.108596"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.090430+0000","last_change":"2026-02-21T14:58:49.096029+0000","last_active":"2026-02-21T14:58:49.090430+0000","last_peered":"2026-02-21T14:58:49.090430+0000","last_clean":"2026-02-21T14:58:49.090430+0000","last_became_active":"2026-02-21T14:58:47.128560+0000","last_became_peered":"2026-02-21T14:58:47.128560+0000","last_unstale":"2026-02-21T14:58:49.090430+0000","last_undegraded":"2026-02-21T14:58:49.090430+0000","last_fullsized":"2026-02-21T14:58:49.090430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.091993+0000","last_change":"2026-02-21T14:58:49.095614+0000","last_active":"2026-02-21T14:58:49.091993+0000","last_peered":"2026-02-21T14:58:49.091993+0000","last_clean":"2026-02-21T14:58:49.091993+0000","last_became_active":"2026-02-21T14:58:47.125613+0000","last_became_peered":"2026-02-21T14:58:47.125613+0000","last_unstale":"2026-02-21T14:58:49.091993+0000","last_undegraded":"2026-02-21T14:58:49.091993+0000","last_fullsized":"2026-02-21T14:58:49.091993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.091965+0000","last_change":"2026-02-21T14:58:49.098251+0000","last_active":"2026-02-21T14:58:49.091965+0000","last_peered":"2026-02-21T14:58:49.091965+0000","last_clean":"2026-02-21T14:58:49.091965+0000","last_became_active":"2026-02-21T14:58:47.134964+0000","last_became_peered":"2026-02-21T14:58:47.134964+0000","last_unstale":"2026-02-21T14:58:49.091965+0000","last_undegraded":"2026-02-21T14:58:49.091965+0000","last_fullsized":"2026-02-21T14:58:49.091965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.092255+0000","last_change":"2026-02-21T14:58:49.097230+0000","last_active":"2026-02-21T14:58:49.092255+0000","last_peered":"2026-02-21T14:58:49.092255+0000","last_clean":"2026-02-21T14:58:49.092255+0000","last_became_active":"2026-02-21T14:58:47.138164+0000","last_became_peered":"2026-02-21T14:58:47.138164+0000","last_unstale":"2026-02-21T14:58:49.092255+0000","last_undegraded":"2026-02-21T14:58:49.092255+0000","last_fullsized":"2026-02-21T14:58:49.092255+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.097095+0000","last_change":"2026-02-21T14:58:49.098467+0000","last_active":"2026-02-21T14:58:49.097095+0000","last_peered":"2026-02-21T14:58:49.097095+0000","last_clean":"2026-02-21T14:58:49.097095+0000","last_became_active":"2026-02-21T14:58:47.153243+0000","last_became_peered":"2026-02-21T14:58:47.153243+0000","last_unstale":"2026-02-21T14:58:49.097095+0000","last_undegraded":"2026-02-21T14:58:49.097095+0000","last_fullsized":"2026-02-21T14:58:49.097095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.089931+0000","last_change":"2026-02-21T14:58:49.093347+0000","last_active":"2026-02-21T14:58:49.089931+0000","last_peered":"2026-02-21T14:58:49.089931+0000","last_clean":"2026-02-21T14:58:49.089931+0000","last_became_active":"2026-02-21T14:58:47.146202+0000","last_became_peered":"2026-02-21T14:58:47.146202+0000","last_unstale":"2026-02-21T14:58:49.089931+0000","last_undegraded":"2026-02-21T14:58:49.089931+0000","last_fullsized":"2026-02-21T14:58:49.089931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.088015+0000","last_change":"2026-02-21T14:58:49.092137+0000","last_active":"2026-02-21T14:58:49.088015+0000","last_peered":"2026-02-21T14:58:49.088015+0000","last_clean":"2026-02-21T14:58:49.088015+0000","last_became_active":"2026-02-21T14:58:47.160042+0000","last_became_peered":"2026-02-21T14:58:47.160042+0000","last_unstale":"2026-02-21T14:58:49.088015+0000","last_undegraded":"2026-02-21T14:58:49.088015+0000","last_fullsized":"2026-02-21T14:58:49.088015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.092216+0000","last_change":"2026-02-21T14:58:49.098217+0000","last_active":"2026-02-21T14:58:49.092216+0000","last_peered":"2026-02-21T14:58:49.092216+0000","last_clean":"2026-02-21T14:58:49.092216+0000","last_became_active":"2026-02-21T14:58:47.124462+0000","last_became_peered":"2026-02-21T14:58:47.124462+0000","last_unstale":"2026-02-21T14:58:49.092216+0000","last_undegraded":"2026-02-21T14:58:49.092216+0000","last_fullsized":"2026-02-21T14:58:49.092216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T14:58:49.089339+0000","last_change":"2026-02-21T14:58:44.102806+0000","last_active":"2026-02-21T14:58:49.089339+0000","last_peered":"2026-02-21T14:58:49.089339+0000","last_clean":"2026-02-21T14:58:49.089339+0000","last_became_active":"2026-02-21T14:58:44.090205+0000","last_became_peered":"2026-02-21T14:58:44.090205+0000","last_unstale":"2026-02-21T14:58:49.089339+0000","last_undegraded":"2026-02-21T14:58:49.089339+0000","last_fullsized":"2026-02-21T14:58:49.089339+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_clean_scrub_stamp":"2026-02-21T14:58:43.041461+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253366272,"internally_reserved":0,"allocated":2600960,"data_stored":2600960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":316,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166340,"statfs":{"total":53418655744,"available":53418332160,"internally_reserved":0,"allocated":323584,"data_stored":323584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1076,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165580,"statfs":{"total":53418655744,"available":53417553920,"internally_reserved":0,"allocated":1101824,"data_stored":1101824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1148,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165508,"statfs":{"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417553920,"internally_reserved":0,"allocated":1101824,"data_stored":1101824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418332160,"internally_reserved":0,"allocated":323584,"data_stored":323584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T14:58:52.049 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T14:58:52.049 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-21T14:58:52.049 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-21T14:58:52.050 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-21T14:58:52.126 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:52.132+0000 7f2601ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:52.128 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:58:52.136+0000 7f2601ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:58:52.267 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:58:52.267 INFO:teuthology.orchestra.run.trial130.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-21T14:58:52.277 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-21T14:58:52.277 INFO:teuthology.run_tasks:Running task workunit... 2026-02-21T14:58:52.302 INFO:tasks.workunit:Pulling workunits from ref c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f 2026-02-21T14:58:52.302 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-21T14:58:52.303 DEBUG:teuthology.orchestra.run.trial130:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-21T14:58:52.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T14:58:52.320 INFO:teuthology.orchestra.run.trial130.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-21T14:58:52.321 DEBUG:teuthology.orchestra.run.trial130:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-21T14:58:52.380 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-21T14:58:52.380 DEBUG:teuthology.orchestra.run.trial130:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-21T14:58:52.436 INFO:tasks.workunit:timeout=3h 2026-02-21T14:58:52.436 INFO:tasks.workunit:cleanup=True 2026-02-21T14:58:52.437 DEBUG:teuthology.orchestra.run.trial130:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/xxhdx1985126/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f 2026-02-21T14:58:52.499 INFO:tasks.workunit.client.0.trial130.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-21T14:59:10.995 INFO:tasks.workunit.client.0.trial130.stderr:Note: switching to 'c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f'. 2026-02-21T14:59:10.996 INFO:tasks.workunit.client.0.trial130.stderr: 2026-02-21T14:59:10.996 INFO:tasks.workunit.client.0.trial130.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-21T14:59:10.996 INFO:tasks.workunit.client.0.trial130.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-21T14:59:10.996 INFO:tasks.workunit.client.0.trial130.stderr:state without impacting any branches by switching back to a branch. 2026-02-21T14:59:10.996 INFO:tasks.workunit.client.0.trial130.stderr: 2026-02-21T14:59:10.996 INFO:tasks.workunit.client.0.trial130.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-21T14:59:10.996 INFO:tasks.workunit.client.0.trial130.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-21T14:59:10.996 INFO:tasks.workunit.client.0.trial130.stderr: 2026-02-21T14:59:10.996 INFO:tasks.workunit.client.0.trial130.stderr: git switch -c 2026-02-21T14:59:10.997 INFO:tasks.workunit.client.0.trial130.stderr: 2026-02-21T14:59:10.997 INFO:tasks.workunit.client.0.trial130.stderr:Or undo this operation with: 2026-02-21T14:59:10.997 INFO:tasks.workunit.client.0.trial130.stderr: 2026-02-21T14:59:10.997 INFO:tasks.workunit.client.0.trial130.stderr: git switch - 2026-02-21T14:59:10.997 INFO:tasks.workunit.client.0.trial130.stderr: 2026-02-21T14:59:10.997 INFO:tasks.workunit.client.0.trial130.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-21T14:59:10.997 INFO:tasks.workunit.client.0.trial130.stderr: 2026-02-21T14:59:10.997 INFO:tasks.workunit.client.0.trial130.stderr:HEAD is now at c8fda9e9b7a qa/suites/crimson-rados/thrash: crimson cpu num: 1 2026-02-21T14:59:11.001 DEBUG:teuthology.orchestra.run.trial130:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-21T14:59:11.059 INFO:tasks.workunit.client.0.trial130.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-21T14:59:11.061 INFO:tasks.workunit.client.0.trial130.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-21T14:59:11.061 INFO:tasks.workunit.client.0.trial130.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-21T14:59:11.152 INFO:tasks.workunit.client.0.trial130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-21T14:59:11.183 INFO:tasks.workunit.client.0.trial130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-21T14:59:11.208 INFO:tasks.workunit.client.0.trial130.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-21T14:59:11.210 INFO:tasks.workunit.client.0.trial130.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-21T14:59:11.210 INFO:tasks.workunit.client.0.trial130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-21T14:59:11.236 INFO:tasks.workunit.client.0.trial130.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-21T14:59:11.240 DEBUG:teuthology.orchestra.run.trial130:> set -ex 2026-02-21T14:59:11.240 DEBUG:teuthology.orchestra.run.trial130:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-21T14:59:11.297 INFO:tasks.workunit:Running workunits matching rbd/test_lock_fence.sh on client.0... 2026-02-21T14:59:11.300 INFO:tasks.workunit:Running workunit rbd/test_lock_fence.sh... 2026-02-21T14:59:11.300 DEBUG:teuthology.orchestra.run.trial130:workunit test rbd/test_lock_fence.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_lock_fence.sh 2026-02-21T14:59:11.357 INFO:tasks.workunit.client.0.trial130.stderr:+ IMAGE=rbdrw-image 2026-02-21T14:59:11.358 INFO:tasks.workunit.client.0.trial130.stderr:+ LOCKID=rbdrw 2026-02-21T14:59:11.358 INFO:tasks.workunit.client.0.trial130.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_lock_fence.sh 2026-02-21T14:59:11.358 INFO:tasks.workunit.client.0.trial130.stderr:+ RELPATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/librbd 2026-02-21T14:59:11.359 INFO:tasks.workunit.client.0.trial130.stderr:+ RBDRW=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/librbd/rbdrw.py 2026-02-21T14:59:11.359 INFO:tasks.workunit.client.0.trial130.stderr:+ rbd create rbdrw-image --size 10 --image-format 2 --image-shared 2026-02-21T14:59:11.385 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:11.394+0000 7fc5700eb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:11.386 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:11.394+0000 7fc5700eb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:11.389 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:11.397+0000 7fc5700eb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:11.465 INFO:tasks.workunit.client.0.trial130.stderr:+ iochild=41752 2026-02-21T14:59:11.466 INFO:tasks.workunit.client.0.trial130.stderr:+ LOCKS='[]' 2026-02-21T14:59:11.466 INFO:tasks.workunit.client.0.trial130.stderr:+ '[' '[]' == '[]' ']' 2026-02-21T14:59:11.466 INFO:tasks.workunit.client.0.trial130.stderr:+ python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/librbd/rbdrw.py rbdrw-image rbdrw 2026-02-21T14:59:11.466 INFO:tasks.workunit.client.0.trial130.stderr:++ rbd lock list rbdrw-image --format json 2026-02-21T14:59:11.493 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:11.501+0000 7f5bc1cfe240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:11.493 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:11.501+0000 7f5bc1cfe240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:11.496 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:11.504+0000 7f5bc1cfe240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:11.635 INFO:tasks.workunit.client.0.trial130.stderr:+ LOCKS='[]' 2026-02-21T14:59:11.635 INFO:tasks.workunit.client.0.trial130.stderr:+ sleep 1 2026-02-21T14:59:12.636 INFO:tasks.workunit.client.0.trial130.stderr:+ '[' '[]' == '[]' ']' 2026-02-21T14:59:12.636 INFO:tasks.workunit.client.0.trial130.stderr:++ rbd lock list rbdrw-image --format json 2026-02-21T14:59:12.663 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:12.671+0000 7fac3f67f240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:12.664 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:12.671+0000 7fac3f67f240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:12.666 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:12.675+0000 7fac3f67f240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:12.803 INFO:tasks.workunit.client.0.trial130.stderr:+ LOCKS='[{"id":"rbdrw","locker":"client.4207","address":"10.20.193.130:0/2325844733"}]' 2026-02-21T14:59:12.803 INFO:tasks.workunit.client.0.trial130.stderr:+ sleep 1 2026-02-21T14:59:13.804 INFO:tasks.workunit.client.0.trial130.stderr:+ '[' '[{"id":"rbdrw","locker":"client.4207","address":"10.20.193.130:0/2325844733"}]' == '[]' ']' 2026-02-21T14:59:13.804 INFO:tasks.workunit.client.0.trial130.stderr:++ rbd lock list rbdrw-image 2026-02-21T14:59:13.804 INFO:tasks.workunit.client.0.trial130.stderr:++ tail -1 2026-02-21T14:59:13.804 INFO:tasks.workunit.client.0.trial130.stderr:++ awk '{print $NF;}' 2026-02-21T14:59:13.832 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:13.840+0000 7f754e5cb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:13.832 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:13.840+0000 7f754e5cb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:13.835 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:13.843+0000 7f754e5cb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:13.978 INFO:tasks.workunit.client.0.trial130.stderr:+ clientaddr=10.20.193.130:0/2325844733 2026-02-21T14:59:13.978 INFO:tasks.workunit.client.0.trial130.stderr:++ rbd lock list rbdrw-image 2026-02-21T14:59:13.979 INFO:tasks.workunit.client.0.trial130.stderr:++ tail -1 2026-02-21T14:59:13.979 INFO:tasks.workunit.client.0.trial130.stderr:++ awk '{print $1;}' 2026-02-21T14:59:14.005 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:14.013+0000 7f1a1b278240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:14.005 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:14.013+0000 7f1a1b278240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:14.009 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:14.017+0000 7f1a1b278240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:14.147 INFO:tasks.workunit.client.0.trial130.stderr:+ clientid=client.4207 2026-02-21T14:59:14.148 INFO:tasks.workunit.client.0.trial130.stdout:clientaddr: 10.20.193.130:0/2325844733 2026-02-21T14:59:14.148 INFO:tasks.workunit.client.0.trial130.stdout:clientid: client.4207 2026-02-21T14:59:14.148 INFO:tasks.workunit.client.0.trial130.stderr:+ echo 'clientaddr: 10.20.193.130:0/2325844733' 2026-02-21T14:59:14.148 INFO:tasks.workunit.client.0.trial130.stderr:+ echo 'clientid: client.4207' 2026-02-21T14:59:14.148 INFO:tasks.workunit.client.0.trial130.stderr:+ ceph osd blocklist add 10.20.193.130:0/2325844733 2026-02-21T14:59:14.208 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:14.212+0000 7fa80c051640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:14.210 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:14.218+0000 7fa80c051640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:16.227 INFO:tasks.workunit.client.0.trial130.stderr:blocklisting 10.20.193.130:0/2325844733 until 2026-02-21T15:59:15.305919+0000 (3600 sec) 2026-02-21T14:59:16.239 INFO:tasks.workunit.client.0.trial130.stderr:+ wait 41752 2026-02-21T14:59:16.239 INFO:tasks.workunit.client.0.trial130.stderr:+ rbdrw_exitcode=108 2026-02-21T14:59:16.239 INFO:tasks.workunit.client.0.trial130.stderr:+ '[' 108 '!=' 108 ']' 2026-02-21T14:59:16.240 INFO:tasks.workunit.client.0.trial130.stdout:rbdrw stopped with ESHUTDOWN 2026-02-21T14:59:16.240 INFO:tasks.workunit.client.0.trial130.stderr:+ echo 'rbdrw stopped with ESHUTDOWN' 2026-02-21T14:59:16.240 INFO:tasks.workunit.client.0.trial130.stderr:+ set -e 2026-02-21T14:59:16.240 INFO:tasks.workunit.client.0.trial130.stderr:+ ceph osd blocklist rm 10.20.193.130:0/2325844733 2026-02-21T14:59:16.295 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:16.300+0000 7fb275a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:16.298 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:16.306+0000 7fb275a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:17.298 INFO:tasks.workunit.client.0.trial130.stderr:10.20.193.130:0/2325844733 isn't blocklisted 2026-02-21T14:59:17.307 INFO:tasks.workunit.client.0.trial130.stderr:+ rbd lock remove rbdrw-image rbdrw client.4207 2026-02-21T14:59:17.334 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:17.342+0000 7f0fb68df240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:17.334 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:17.342+0000 7f0fb68df240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:17.337 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:17.346+0000 7f0fb68df240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:18.289 INFO:tasks.workunit.client.0.trial130.stderr:+ sleep 30 2026-02-21T14:59:48.290 INFO:tasks.workunit.client.0.trial130.stderr:+ rbd rm rbdrw-image 2026-02-21T14:59:48.317 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:48.325+0000 7f1d3be15240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:48.317 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:48.325+0000 7f1d3be15240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:48.320 INFO:tasks.workunit.client.0.trial130.stderr:2026-02-21T14:59:48.328+0000 7f1d3be15240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:48.831 INFO:tasks.workunit.client.0.trial130.stderr: Removing image: 33% complete... Removing image: 66% complete... Removing image: 100% complete...done. 2026-02-21T14:59:48.835 INFO:tasks.workunit.client.0.trial130.stderr:+ echo OK 2026-02-21T14:59:48.836 INFO:tasks.workunit.client.0.trial130.stdout:OK 2026-02-21T14:59:48.836 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-21T14:59:48.837 DEBUG:teuthology.orchestra.run.trial130:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T14:59:48.903 INFO:tasks.workunit:Stopping ['rbd/test_lock_fence.sh'] on client.0... 2026-02-21T14:59:48.903 DEBUG:teuthology.orchestra.run.trial130:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-21T14:59:49.270 DEBUG:teuthology.parallel:result is None 2026-02-21T14:59:49.270 DEBUG:teuthology.orchestra.run.trial130:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T14:59:49.295 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T14:59:49.296 DEBUG:teuthology.orchestra.run.trial130:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-21T14:59:49.352 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T14:59:49.353 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-21T14:59:49.370 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T14:59:49.371 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T14:59:49.469 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:49.475+0000 7f6213ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:49.471 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:49.479+0000 7f6213ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:49.584 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:59:49.584 INFO:teuthology.orchestra.run.trial130.stderr:dumped all 2026-02-21T14:59:49.593 INFO:teuthology.orchestra.run.trial130.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2026-02-21T14:59:48.180833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594499,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":445,"num_read_kb":913,"num_write":295,"num_write_kb":1947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":3260,"kb_used_data":3260,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156496708,"statfs":{"total":160255967232,"available":160252628992,"internally_reserved":0,"allocated":3338240,"data_stored":3338240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002489"},"pg_stats":[{"pgid":"2.7","version":"21'6","reported_seq":91,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.283683+0000","last_change":"2026-02-21T14:58:49.096029+0000","last_active":"2026-02-21T14:59:18.283683+0000","last_peered":"2026-02-21T14:59:18.283683+0000","last_clean":"2026-02-21T14:59:18.283683+0000","last_became_active":"2026-02-21T14:58:47.128560+0000","last_became_peered":"2026-02-21T14:58:47.128560+0000","last_unstale":"2026-02-21T14:59:18.283683+0000","last_undegraded":"2026-02-21T14:59:18.283683+0000","last_fullsized":"2026-02-21T14:59:18.283683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":161,"num_read_kb":28,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262999+0000","last_change":"2026-02-21T14:58:49.095614+0000","last_active":"2026-02-21T14:59:18.262999+0000","last_peered":"2026-02-21T14:59:18.262999+0000","last_clean":"2026-02-21T14:59:18.262999+0000","last_became_active":"2026-02-21T14:58:47.125613+0000","last_became_peered":"2026-02-21T14:58:47.125613+0000","last_unstale":"2026-02-21T14:59:18.262999+0000","last_undegraded":"2026-02-21T14:59:18.262999+0000","last_fullsized":"2026-02-21T14:59:18.262999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"17'1","reported_seq":40,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.263191+0000","last_change":"2026-02-21T14:58:49.098251+0000","last_active":"2026-02-21T14:59:18.263191+0000","last_peered":"2026-02-21T14:59:18.263191+0000","last_clean":"2026-02-21T14:59:18.263191+0000","last_became_active":"2026-02-21T14:58:47.134964+0000","last_became_peered":"2026-02-21T14:58:47.134964+0000","last_unstale":"2026-02-21T14:59:18.263191+0000","last_undegraded":"2026-02-21T14:59:18.263191+0000","last_fullsized":"2026-02-21T14:59:18.263191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.263101+0000","last_change":"2026-02-21T14:58:49.097230+0000","last_active":"2026-02-21T14:59:18.263101+0000","last_peered":"2026-02-21T14:59:18.263101+0000","last_clean":"2026-02-21T14:59:18.263101+0000","last_became_active":"2026-02-21T14:58:47.138164+0000","last_became_peered":"2026-02-21T14:58:47.138164+0000","last_unstale":"2026-02-21T14:59:18.263101+0000","last_undegraded":"2026-02-21T14:59:18.263101+0000","last_fullsized":"2026-02-21T14:59:18.263101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":38,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.264852+0000","last_change":"2026-02-21T14:58:49.098467+0000","last_active":"2026-02-21T14:59:18.264852+0000","last_peered":"2026-02-21T14:59:18.264852+0000","last_clean":"2026-02-21T14:59:18.264852+0000","last_became_active":"2026-02-21T14:58:47.153243+0000","last_became_peered":"2026-02-21T14:58:47.153243+0000","last_unstale":"2026-02-21T14:59:18.264852+0000","last_undegraded":"2026-02-21T14:59:18.264852+0000","last_fullsized":"2026-02-21T14:59:18.264852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262034+0000","last_change":"2026-02-21T14:58:49.093347+0000","last_active":"2026-02-21T14:59:18.262034+0000","last_peered":"2026-02-21T14:59:18.262034+0000","last_clean":"2026-02-21T14:59:18.262034+0000","last_became_active":"2026-02-21T14:58:47.146202+0000","last_became_peered":"2026-02-21T14:58:47.146202+0000","last_unstale":"2026-02-21T14:59:18.262034+0000","last_undegraded":"2026-02-21T14:59:18.262034+0000","last_fullsized":"2026-02-21T14:59:18.262034+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"17'209","reported_seq":444,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.261458+0000","last_change":"2026-02-21T14:58:49.092137+0000","last_active":"2026-02-21T14:59:18.261458+0000","last_peered":"2026-02-21T14:59:18.261458+0000","last_clean":"2026-02-21T14:59:18.261458+0000","last_became_active":"2026-02-21T14:58:47.160042+0000","last_became_peered":"2026-02-21T14:58:47.160042+0000","last_unstale":"2026-02-21T14:59:18.261458+0000","last_undegraded":"2026-02-21T14:59:18.261458+0000","last_fullsized":"2026-02-21T14:59:18.261458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":832,"num_write":209,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":28,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.263520+0000","last_change":"2026-02-21T14:58:49.098217+0000","last_active":"2026-02-21T14:59:18.263520+0000","last_peered":"2026-02-21T14:59:18.263520+0000","last_clean":"2026-02-21T14:59:18.263520+0000","last_became_active":"2026-02-21T14:58:47.124462+0000","last_became_peered":"2026-02-21T14:58:47.124462+0000","last_unstale":"2026-02-21T14:59:18.263520+0000","last_undegraded":"2026-02-21T14:59:18.263520+0000","last_fullsized":"2026-02-21T14:59:18.263520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262040+0000","last_change":"2026-02-21T14:58:44.102806+0000","last_active":"2026-02-21T14:59:18.262040+0000","last_peered":"2026-02-21T14:59:18.262040+0000","last_clean":"2026-02-21T14:59:18.262040+0000","last_became_active":"2026-02-21T14:58:44.090205+0000","last_became_peered":"2026-02-21T14:58:44.090205+0000","last_unstale":"2026-02-21T14:59:18.262040+0000","last_undegraded":"2026-02-21T14:59:18.262040+0000","last_fullsized":"2026-02-21T14:59:18.262040+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_clean_scrub_stamp":"2026-02-21T14:58:43.041461+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":4131,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":392,"num_read_kb":867,"num_write":216,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160252628992,"internally_reserved":0,"allocated":3338240,"data_stored":3338240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949673024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":464,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166192,"statfs":{"total":53418655744,"available":53418180608,"internally_reserved":0,"allocated":475136,"data_stored":475136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1328,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165328,"statfs":{"total":53418655744,"available":53417295872,"internally_reserved":0,"allocated":1359872,"data_stored":1359872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705728,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1468,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165188,"statfs":{"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417295872,"internally_reserved":0,"allocated":1359872,"data_stored":1359872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418180608,"internally_reserved":0,"allocated":475136,"data_stored":475136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T14:59:49.595 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T14:59:49.672 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:49.678+0000 7f3e545fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:49.675 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:49.683+0000 7f3e545fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:49.788 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:59:49.788 INFO:teuthology.orchestra.run.trial130.stderr:dumped all 2026-02-21T14:59:49.797 INFO:teuthology.orchestra.run.trial130.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2026-02-21T14:59:48.180833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594499,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":445,"num_read_kb":913,"num_write":295,"num_write_kb":1947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":3260,"kb_used_data":3260,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156496708,"statfs":{"total":160255967232,"available":160252628992,"internally_reserved":0,"allocated":3338240,"data_stored":3338240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002489"},"pg_stats":[{"pgid":"2.7","version":"21'6","reported_seq":91,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.283683+0000","last_change":"2026-02-21T14:58:49.096029+0000","last_active":"2026-02-21T14:59:18.283683+0000","last_peered":"2026-02-21T14:59:18.283683+0000","last_clean":"2026-02-21T14:59:18.283683+0000","last_became_active":"2026-02-21T14:58:47.128560+0000","last_became_peered":"2026-02-21T14:58:47.128560+0000","last_unstale":"2026-02-21T14:59:18.283683+0000","last_undegraded":"2026-02-21T14:59:18.283683+0000","last_fullsized":"2026-02-21T14:59:18.283683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":161,"num_read_kb":28,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262999+0000","last_change":"2026-02-21T14:58:49.095614+0000","last_active":"2026-02-21T14:59:18.262999+0000","last_peered":"2026-02-21T14:59:18.262999+0000","last_clean":"2026-02-21T14:59:18.262999+0000","last_became_active":"2026-02-21T14:58:47.125613+0000","last_became_peered":"2026-02-21T14:58:47.125613+0000","last_unstale":"2026-02-21T14:59:18.262999+0000","last_undegraded":"2026-02-21T14:59:18.262999+0000","last_fullsized":"2026-02-21T14:59:18.262999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"17'1","reported_seq":40,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.263191+0000","last_change":"2026-02-21T14:58:49.098251+0000","last_active":"2026-02-21T14:59:18.263191+0000","last_peered":"2026-02-21T14:59:18.263191+0000","last_clean":"2026-02-21T14:59:18.263191+0000","last_became_active":"2026-02-21T14:58:47.134964+0000","last_became_peered":"2026-02-21T14:58:47.134964+0000","last_unstale":"2026-02-21T14:59:18.263191+0000","last_undegraded":"2026-02-21T14:59:18.263191+0000","last_fullsized":"2026-02-21T14:59:18.263191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.263101+0000","last_change":"2026-02-21T14:58:49.097230+0000","last_active":"2026-02-21T14:59:18.263101+0000","last_peered":"2026-02-21T14:59:18.263101+0000","last_clean":"2026-02-21T14:59:18.263101+0000","last_became_active":"2026-02-21T14:58:47.138164+0000","last_became_peered":"2026-02-21T14:58:47.138164+0000","last_unstale":"2026-02-21T14:59:18.263101+0000","last_undegraded":"2026-02-21T14:59:18.263101+0000","last_fullsized":"2026-02-21T14:59:18.263101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":38,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.264852+0000","last_change":"2026-02-21T14:58:49.098467+0000","last_active":"2026-02-21T14:59:18.264852+0000","last_peered":"2026-02-21T14:59:18.264852+0000","last_clean":"2026-02-21T14:59:18.264852+0000","last_became_active":"2026-02-21T14:58:47.153243+0000","last_became_peered":"2026-02-21T14:58:47.153243+0000","last_unstale":"2026-02-21T14:59:18.264852+0000","last_undegraded":"2026-02-21T14:59:18.264852+0000","last_fullsized":"2026-02-21T14:59:18.264852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262034+0000","last_change":"2026-02-21T14:58:49.093347+0000","last_active":"2026-02-21T14:59:18.262034+0000","last_peered":"2026-02-21T14:59:18.262034+0000","last_clean":"2026-02-21T14:59:18.262034+0000","last_became_active":"2026-02-21T14:58:47.146202+0000","last_became_peered":"2026-02-21T14:58:47.146202+0000","last_unstale":"2026-02-21T14:59:18.262034+0000","last_undegraded":"2026-02-21T14:59:18.262034+0000","last_fullsized":"2026-02-21T14:59:18.262034+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"17'209","reported_seq":444,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.261458+0000","last_change":"2026-02-21T14:58:49.092137+0000","last_active":"2026-02-21T14:59:18.261458+0000","last_peered":"2026-02-21T14:59:18.261458+0000","last_clean":"2026-02-21T14:59:18.261458+0000","last_became_active":"2026-02-21T14:58:47.160042+0000","last_became_peered":"2026-02-21T14:58:47.160042+0000","last_unstale":"2026-02-21T14:59:18.261458+0000","last_undegraded":"2026-02-21T14:59:18.261458+0000","last_fullsized":"2026-02-21T14:59:18.261458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":832,"num_write":209,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":28,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.263520+0000","last_change":"2026-02-21T14:58:49.098217+0000","last_active":"2026-02-21T14:59:18.263520+0000","last_peered":"2026-02-21T14:59:18.263520+0000","last_clean":"2026-02-21T14:59:18.263520+0000","last_became_active":"2026-02-21T14:58:47.124462+0000","last_became_peered":"2026-02-21T14:58:47.124462+0000","last_unstale":"2026-02-21T14:59:18.263520+0000","last_undegraded":"2026-02-21T14:59:18.263520+0000","last_fullsized":"2026-02-21T14:59:18.263520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262040+0000","last_change":"2026-02-21T14:58:44.102806+0000","last_active":"2026-02-21T14:59:18.262040+0000","last_peered":"2026-02-21T14:59:18.262040+0000","last_clean":"2026-02-21T14:59:18.262040+0000","last_became_active":"2026-02-21T14:58:44.090205+0000","last_became_peered":"2026-02-21T14:58:44.090205+0000","last_unstale":"2026-02-21T14:59:18.262040+0000","last_undegraded":"2026-02-21T14:59:18.262040+0000","last_fullsized":"2026-02-21T14:59:18.262040+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_clean_scrub_stamp":"2026-02-21T14:58:43.041461+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":4131,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":392,"num_read_kb":867,"num_write":216,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160252628992,"internally_reserved":0,"allocated":3338240,"data_stored":3338240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949673024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":464,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166192,"statfs":{"total":53418655744,"available":53418180608,"internally_reserved":0,"allocated":475136,"data_stored":475136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1328,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165328,"statfs":{"total":53418655744,"available":53417295872,"internally_reserved":0,"allocated":1359872,"data_stored":1359872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705728,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1468,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165188,"statfs":{"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417295872,"internally_reserved":0,"allocated":1359872,"data_stored":1359872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418180608,"internally_reserved":0,"allocated":475136,"data_stored":475136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T14:59:49.798 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T14:59:49.798 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T14:59:49.877 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:49.884+0000 7fcdf703d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:49.880 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:49.888+0000 7fcdf703d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:49.992 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:59:49.993 INFO:teuthology.orchestra.run.trial130.stderr:dumped all 2026-02-21T14:59:50.003 INFO:teuthology.orchestra.run.trial130.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2026-02-21T14:59:48.180833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594499,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":445,"num_read_kb":913,"num_write":295,"num_write_kb":1947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":3260,"kb_used_data":3260,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156496708,"statfs":{"total":160255967232,"available":160252628992,"internally_reserved":0,"allocated":3338240,"data_stored":3338240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002489"},"pg_stats":[{"pgid":"2.7","version":"21'6","reported_seq":91,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.283683+0000","last_change":"2026-02-21T14:58:49.096029+0000","last_active":"2026-02-21T14:59:18.283683+0000","last_peered":"2026-02-21T14:59:18.283683+0000","last_clean":"2026-02-21T14:59:18.283683+0000","last_became_active":"2026-02-21T14:58:47.128560+0000","last_became_peered":"2026-02-21T14:58:47.128560+0000","last_unstale":"2026-02-21T14:59:18.283683+0000","last_undegraded":"2026-02-21T14:59:18.283683+0000","last_fullsized":"2026-02-21T14:59:18.283683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":161,"num_read_kb":28,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262999+0000","last_change":"2026-02-21T14:58:49.095614+0000","last_active":"2026-02-21T14:59:18.262999+0000","last_peered":"2026-02-21T14:59:18.262999+0000","last_clean":"2026-02-21T14:59:18.262999+0000","last_became_active":"2026-02-21T14:58:47.125613+0000","last_became_peered":"2026-02-21T14:58:47.125613+0000","last_unstale":"2026-02-21T14:59:18.262999+0000","last_undegraded":"2026-02-21T14:59:18.262999+0000","last_fullsized":"2026-02-21T14:59:18.262999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"17'1","reported_seq":40,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.263191+0000","last_change":"2026-02-21T14:58:49.098251+0000","last_active":"2026-02-21T14:59:18.263191+0000","last_peered":"2026-02-21T14:59:18.263191+0000","last_clean":"2026-02-21T14:59:18.263191+0000","last_became_active":"2026-02-21T14:58:47.134964+0000","last_became_peered":"2026-02-21T14:58:47.134964+0000","last_unstale":"2026-02-21T14:59:18.263191+0000","last_undegraded":"2026-02-21T14:59:18.263191+0000","last_fullsized":"2026-02-21T14:59:18.263191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.263101+0000","last_change":"2026-02-21T14:58:49.097230+0000","last_active":"2026-02-21T14:59:18.263101+0000","last_peered":"2026-02-21T14:59:18.263101+0000","last_clean":"2026-02-21T14:59:18.263101+0000","last_became_active":"2026-02-21T14:58:47.138164+0000","last_became_peered":"2026-02-21T14:58:47.138164+0000","last_unstale":"2026-02-21T14:59:18.263101+0000","last_undegraded":"2026-02-21T14:59:18.263101+0000","last_fullsized":"2026-02-21T14:59:18.263101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":38,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.264852+0000","last_change":"2026-02-21T14:58:49.098467+0000","last_active":"2026-02-21T14:59:18.264852+0000","last_peered":"2026-02-21T14:59:18.264852+0000","last_clean":"2026-02-21T14:59:18.264852+0000","last_became_active":"2026-02-21T14:58:47.153243+0000","last_became_peered":"2026-02-21T14:58:47.153243+0000","last_unstale":"2026-02-21T14:59:18.264852+0000","last_undegraded":"2026-02-21T14:59:18.264852+0000","last_fullsized":"2026-02-21T14:59:18.264852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262034+0000","last_change":"2026-02-21T14:58:49.093347+0000","last_active":"2026-02-21T14:59:18.262034+0000","last_peered":"2026-02-21T14:59:18.262034+0000","last_clean":"2026-02-21T14:59:18.262034+0000","last_became_active":"2026-02-21T14:58:47.146202+0000","last_became_peered":"2026-02-21T14:58:47.146202+0000","last_unstale":"2026-02-21T14:59:18.262034+0000","last_undegraded":"2026-02-21T14:59:18.262034+0000","last_fullsized":"2026-02-21T14:59:18.262034+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"17'209","reported_seq":444,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.261458+0000","last_change":"2026-02-21T14:58:49.092137+0000","last_active":"2026-02-21T14:59:18.261458+0000","last_peered":"2026-02-21T14:59:18.261458+0000","last_clean":"2026-02-21T14:59:18.261458+0000","last_became_active":"2026-02-21T14:58:47.160042+0000","last_became_peered":"2026-02-21T14:58:47.160042+0000","last_unstale":"2026-02-21T14:59:18.261458+0000","last_undegraded":"2026-02-21T14:59:18.261458+0000","last_fullsized":"2026-02-21T14:59:18.261458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":832,"num_write":209,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":28,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.263520+0000","last_change":"2026-02-21T14:58:49.098217+0000","last_active":"2026-02-21T14:59:18.263520+0000","last_peered":"2026-02-21T14:59:18.263520+0000","last_clean":"2026-02-21T14:59:18.263520+0000","last_became_active":"2026-02-21T14:58:47.124462+0000","last_became_peered":"2026-02-21T14:58:47.124462+0000","last_unstale":"2026-02-21T14:59:18.263520+0000","last_undegraded":"2026-02-21T14:59:18.263520+0000","last_fullsized":"2026-02-21T14:59:18.263520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262040+0000","last_change":"2026-02-21T14:58:44.102806+0000","last_active":"2026-02-21T14:59:18.262040+0000","last_peered":"2026-02-21T14:59:18.262040+0000","last_clean":"2026-02-21T14:59:18.262040+0000","last_became_active":"2026-02-21T14:58:44.090205+0000","last_became_peered":"2026-02-21T14:58:44.090205+0000","last_unstale":"2026-02-21T14:59:18.262040+0000","last_undegraded":"2026-02-21T14:59:18.262040+0000","last_fullsized":"2026-02-21T14:59:18.262040+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_clean_scrub_stamp":"2026-02-21T14:58:43.041461+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":4131,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":392,"num_read_kb":867,"num_write":216,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160252628992,"internally_reserved":0,"allocated":3338240,"data_stored":3338240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949673024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":464,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166192,"statfs":{"total":53418655744,"available":53418180608,"internally_reserved":0,"allocated":475136,"data_stored":475136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1328,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165328,"statfs":{"total":53418655744,"available":53417295872,"internally_reserved":0,"allocated":1359872,"data_stored":1359872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705728,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1468,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165188,"statfs":{"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417295872,"internally_reserved":0,"allocated":1359872,"data_stored":1359872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417152512,"internally_reserved":0,"allocated":1503232,"data_stored":1503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418180608,"internally_reserved":0,"allocated":475136,"data_stored":475136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T14:59:50.004 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T14:59:50.081 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:50.087+0000 7f3862f83640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:50.083 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:50.092+0000 7f3862f83640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:50.198 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:59:50.198 INFO:teuthology.orchestra.run.trial130.stdout:{"epoch":21,"fsid":"05ac4a4e-7e84-4852-9cff-af5d19cabadf","created":"2026-02-21T14:58:29.754604+0000","modified":"2026-02-21T14:59:18.243744+0000","last_up_change":"2026-02-21T14:58:41.036030+0000","last_in_change":"2026-02-21T14:58:31.519727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T14:58:42.184030+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T14:58:45.519400+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19de4c87-86a9-4917-8572-358cf6c28af1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6805","nonce":1758314576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6804","nonce":1758314576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6808","nonce":1758314576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6809","nonce":1758314576}]},"public_addr":"10.20.193.130:6805/1758314576","cluster_addr":"10.20.193.130:6804/1758314576","heartbeat_back_addr":"10.20.193.130:6808/1758314576","heartbeat_front_addr":"10.20.193.130:6809/1758314576","state":["exists","up"]},{"osd":1,"uuid":"be1c4136-6a34-402e-88fe-5c6867ae35ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6811","nonce":1995513605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6810","nonce":1995513605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6806","nonce":1995513605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6807","nonce":1995513605}]},"public_addr":"10.20.193.130:6811/1995513605","cluster_addr":"10.20.193.130:6810/1995513605","heartbeat_back_addr":"10.20.193.130:6806/1995513605","heartbeat_front_addr":"10.20.193.130:6807/1995513605","state":["exists","up"]},{"osd":2,"uuid":"6df6fcfe-eebc-4350-a5d3-9d885e6a904a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6803","nonce":2889879952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6802","nonce":2889879952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6817","nonce":2889879952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.130:6816","nonce":2889879952}]},"public_addr":"10.20.193.130:6803/2889879952","cluster_addr":"10.20.193.130:6802/2889879952","heartbeat_back_addr":"10.20.193.130:6817/2889879952","heartbeat_front_addr":"10.20.193.130:6816/2889879952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.130:0/2325844733":"2026-02-21T15:59:17.467591+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T14:59:51.210 INFO:tasks.ceph:Scrubbing osd.0 2026-02-21T14:59:51.210 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-21T14:59:51.288 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:51.292+0000 7ff50c211640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:51.290 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:51.298+0000 7ff50c211640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:51.300 INFO:teuthology.orchestra.run.trial130.stdout:{ 2026-02-21T14:59:51.300 INFO:teuthology.orchestra.run.trial130.stdout: "success": "" 2026-02-21T14:59:51.300 INFO:teuthology.orchestra.run.trial130.stdout:} 2026-02-21T14:59:51.308 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-21T14:59:51.385 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:51.390+0000 7f286da5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:51.387 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:51.395+0000 7f286da5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:51.500 INFO:teuthology.orchestra.run.trial130.stderr:instructed osd(s) 0 to deep-scrub 2026-02-21T14:59:51.511 INFO:tasks.ceph:Scrubbing osd.1 2026-02-21T14:59:51.512 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-21T14:59:51.591 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:51.594+0000 7f0e658cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:51.593 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:51.601+0000 7f0e658cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:51.602 INFO:teuthology.orchestra.run.trial130.stdout:{ 2026-02-21T14:59:51.603 INFO:teuthology.orchestra.run.trial130.stdout: "success": "" 2026-02-21T14:59:51.603 INFO:teuthology.orchestra.run.trial130.stdout:} 2026-02-21T14:59:51.611 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-21T14:59:51.689 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:51.694+0000 7f18d3c51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:51.691 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:51.699+0000 7f18d3c51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:51.804 INFO:teuthology.orchestra.run.trial130.stderr:instructed osd(s) 1 to deep-scrub 2026-02-21T14:59:51.815 INFO:tasks.ceph:Scrubbing osd.2 2026-02-21T14:59:51.815 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-21T14:59:51.896 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:51.899+0000 7f78930cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:51.898 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:51.906+0000 7f78930cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:51.907 INFO:teuthology.orchestra.run.trial130.stdout:{ 2026-02-21T14:59:51.907 INFO:teuthology.orchestra.run.trial130.stdout: "success": "" 2026-02-21T14:59:51.907 INFO:teuthology.orchestra.run.trial130.stdout:} 2026-02-21T14:59:51.914 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-21T14:59:51.991 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:51.996+0000 7fc14525a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:51.993 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:52.001+0000 7fc14525a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:52.110 INFO:teuthology.orchestra.run.trial130.stderr:instructed osd(s) 2 to deep-scrub 2026-02-21T14:59:52.122 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T14:59:52.200 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:52.206+0000 7fe605bcc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:52.202 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T14:59:52.211+0000 7fe605bcc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T14:59:52.315 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T14:59:52.315 INFO:teuthology.orchestra.run.trial130.stderr:dumped all 2026-02-21T14:59:52.325 INFO:teuthology.orchestra.run.trial130.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2026-02-21T14:59:52.181493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":925,"num_write":301,"num_write_kb":1950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":284,"ondisk_log_size":284,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":3148,"kb_used_data":3148,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156496820,"statfs":{"total":160255967232,"available":160252743680,"internally_reserved":0,"allocated":3223552,"data_stored":3223552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4112,"num_objects":-3,"num_object_clones":0,"num_object_copies":-6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":12,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002357"},"pg_stats":[{"pgid":"2.7","version":"21'13","reported_seq":126,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:48.772473+0000","last_change":"2026-02-21T14:58:49.096029+0000","last_active":"2026-02-21T14:59:48.772473+0000","last_peered":"2026-02-21T14:59:48.772473+0000","last_clean":"2026-02-21T14:59:48.772473+0000","last_became_active":"2026-02-21T14:58:47.128560+0000","last_became_peered":"2026-02-21T14:58:47.128560+0000","last_unstale":"2026-02-21T14:59:48.772473+0000","last_undegraded":"2026-02-21T14:59:48.772473+0000","last_fullsized":"2026-02-21T14:59:48.772473+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":235,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262999+0000","last_change":"2026-02-21T14:58:49.095614+0000","last_active":"2026-02-21T14:59:18.262999+0000","last_peered":"2026-02-21T14:59:18.262999+0000","last_clean":"2026-02-21T14:59:18.262999+0000","last_became_active":"2026-02-21T14:58:47.125613+0000","last_became_peered":"2026-02-21T14:58:47.125613+0000","last_unstale":"2026-02-21T14:59:18.262999+0000","last_undegraded":"2026-02-21T14:59:18.262999+0000","last_fullsized":"2026-02-21T14:59:18.262999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"17'1","reported_seq":41,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.263403+0000","last_change":"2026-02-21T14:58:49.098251+0000","last_active":"2026-02-21T14:59:18.263403+0000","last_peered":"2026-02-21T14:59:18.263403+0000","last_clean":"2026-02-21T14:59:18.263403+0000","last_became_active":"2026-02-21T14:58:47.134964+0000","last_became_peered":"2026-02-21T14:58:47.134964+0000","last_unstale":"2026-02-21T14:59:18.263403+0000","last_undegraded":"2026-02-21T14:59:18.263403+0000","last_fullsized":"2026-02-21T14:59:18.263403+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":6,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":33,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:48.471251+0000","last_change":"2026-02-21T14:58:49.097230+0000","last_active":"2026-02-21T14:59:48.471251+0000","last_peered":"2026-02-21T14:59:48.471251+0000","last_clean":"2026-02-21T14:59:48.471251+0000","last_became_active":"2026-02-21T14:58:47.138164+0000","last_became_peered":"2026-02-21T14:58:47.138164+0000","last_unstale":"2026-02-21T14:59:48.471251+0000","last_undegraded":"2026-02-21T14:59:48.471251+0000","last_fullsized":"2026-02-21T14:59:48.471251+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":42,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:48.538321+0000","last_change":"2026-02-21T14:58:49.098467+0000","last_active":"2026-02-21T14:59:48.538321+0000","last_peered":"2026-02-21T14:59:48.538321+0000","last_clean":"2026-02-21T14:59:48.538321+0000","last_became_active":"2026-02-21T14:58:47.153243+0000","last_became_peered":"2026-02-21T14:58:47.153243+0000","last_unstale":"2026-02-21T14:59:48.538321+0000","last_undegraded":"2026-02-21T14:59:48.538321+0000","last_fullsized":"2026-02-21T14:59:48.538321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262034+0000","last_change":"2026-02-21T14:58:49.093347+0000","last_active":"2026-02-21T14:59:18.262034+0000","last_peered":"2026-02-21T14:59:18.262034+0000","last_clean":"2026-02-21T14:59:18.262034+0000","last_became_active":"2026-02-21T14:58:47.146202+0000","last_became_peered":"2026-02-21T14:58:47.146202+0000","last_unstale":"2026-02-21T14:59:18.262034+0000","last_undegraded":"2026-02-21T14:59:18.262034+0000","last_fullsized":"2026-02-21T14:59:18.262034+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"17'209","reported_seq":445,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.261722+0000","last_change":"2026-02-21T14:58:49.092137+0000","last_active":"2026-02-21T14:59:18.261722+0000","last_peered":"2026-02-21T14:59:18.261722+0000","last_clean":"2026-02-21T14:59:18.261722+0000","last_became_active":"2026-02-21T14:58:47.160042+0000","last_became_peered":"2026-02-21T14:58:47.160042+0000","last_unstale":"2026-02-21T14:59:18.261722+0000","last_undegraded":"2026-02-21T14:59:18.261722+0000","last_fullsized":"2026-02-21T14:59:18.261722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":832,"num_write":210,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":36,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:48.829697+0000","last_change":"2026-02-21T14:58:49.098217+0000","last_active":"2026-02-21T14:59:48.829697+0000","last_peered":"2026-02-21T14:59:48.829697+0000","last_clean":"2026-02-21T14:59:48.829697+0000","last_became_active":"2026-02-21T14:58:47.124462+0000","last_became_peered":"2026-02-21T14:58:47.124462+0000","last_unstale":"2026-02-21T14:59:48.829697+0000","last_undegraded":"2026-02-21T14:59:48.829697+0000","last_fullsized":"2026-02-21T14:59:48.829697+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:46.054218+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:18.262040+0000","last_change":"2026-02-21T14:58:44.102806+0000","last_active":"2026-02-21T14:59:18.262040+0000","last_peered":"2026-02-21T14:59:18.262040+0000","last_clean":"2026-02-21T14:59:18.262040+0000","last_became_active":"2026-02-21T14:58:44.090205+0000","last_became_peered":"2026-02-21T14:58:44.090205+0000","last_unstale":"2026-02-21T14:59:18.262040+0000","last_undegraded":"2026-02-21T14:59:18.262040+0000","last_fullsized":"2026-02-21T14:59:18.262040+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:58:43.041461+0000","last_clean_scrub_stamp":"2026-02-21T14:58:43.041461+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":477,"num_read_kb":879,"num_write":222,"num_write_kb":844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160252743680,"internally_reserved":0,"allocated":3223552,"data_stored":3223552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417209856,"internally_reserved":0,"allocated":1445888,"data_stored":1445888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949673029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":540,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166116,"statfs":{"total":53418655744,"available":53418102784,"internally_reserved":0,"allocated":552960,"data_stored":552960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1196,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165460,"statfs":{"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705733,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1412,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165244,"statfs":{"total":53418655744,"available":53417209856,"internally_reserved":0,"allocated":1445888,"data_stored":1445888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417209856,"internally_reserved":0,"allocated":1445888,"data_stored":1445888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417209856,"internally_reserved":0,"allocated":1445888,"data_stored":1445888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418102784,"internally_reserved":0,"allocated":552960,"data_stored":552960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T14:59:52.330 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-21T14:58:46.054218+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=46, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=59, tm_sec=50, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T14:59:52.331 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-21T14:58:46.054218+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=46, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=59, tm_sec=50, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T14:59:52.331 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-21T14:58:46.054218+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=46, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=59, tm_sec=50, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T14:59:52.331 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-21T14:58:46.054218+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=46, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=59, tm_sec=50, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T14:59:52.331 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-21T14:58:46.054218+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=46, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=59, tm_sec=50, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T14:59:52.331 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-21T14:58:46.054218+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=46, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=59, tm_sec=50, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T14:59:52.331 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-21T14:58:46.054218+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=46, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=59, tm_sec=50, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T14:59:52.332 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-21T14:58:46.054218+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=46, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=59, tm_sec=50, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T14:59:52.332 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-21T14:58:43.041461+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=43, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=14, tm_min=59, tm_sec=50, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T14:59:52.332 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-21T15:00:12.332 DEBUG:teuthology.orchestra.run.trial130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T15:00:12.412 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T15:00:12.419+0000 7f175fa5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T15:00:12.415 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T15:00:12.424+0000 7f175fa5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T15:00:12.528 INFO:teuthology.orchestra.run.trial130.stdout: 2026-02-21T15:00:12.528 INFO:teuthology.orchestra.run.trial130.stderr:dumped all 2026-02-21T15:00:12.538 INFO:teuthology.orchestra.run.trial130.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2026-02-21T15:00:12.185476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":925,"num_write":301,"num_write_kb":1950,"num_scrub_errors":0,"num_shallow_scrub_errors":2,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":292,"ondisk_log_size":292,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":3148,"kb_used_data":3148,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156496820,"statfs":{"total":160255967232,"available":160252743680,"internally_reserved":0,"allocated":3223552,"data_stored":3223552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002551"},"pg_stats":[{"pgid":"2.7","version":"21'15","reported_seq":128,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:51.819263+0000","last_change":"2026-02-21T14:59:51.847653+0000","last_active":"2026-02-21T14:59:51.819263+0000","last_peered":"2026-02-21T14:59:51.819263+0000","last_clean":"2026-02-21T14:59:51.819263+0000","last_became_active":"2026-02-21T14:58:47.128560+0000","last_became_peered":"2026-02-21T14:58:47.128560+0000","last_unstale":"2026-02-21T14:59:51.819263+0000","last_undegraded":"2026-02-21T14:59:51.819263+0000","last_fullsized":"2026-02-21T14:59:51.819263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'15","last_scrub_stamp":"2026-02-21T14:59:51.847596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:59:51.847596+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":235,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'2","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:51.818014+0000","last_change":"2026-02-21T14:59:51.836218+0000","last_active":"2026-02-21T14:59:51.818014+0000","last_peered":"2026-02-21T14:59:51.818014+0000","last_clean":"2026-02-21T14:59:51.818014+0000","last_became_active":"2026-02-21T14:58:47.125613+0000","last_became_peered":"2026-02-21T14:58:47.125613+0000","last_unstale":"2026-02-21T14:59:51.818014+0000","last_undegraded":"2026-02-21T14:59:51.818014+0000","last_fullsized":"2026-02-21T14:59:51.818014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'2","last_scrub_stamp":"2026-02-21T14:59:51.836151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:59:51.836151+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"21'2","reported_seq":43,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:51.817932+0000","last_change":"2026-02-21T14:59:51.841898+0000","last_active":"2026-02-21T14:59:51.817932+0000","last_peered":"2026-02-21T14:59:51.817932+0000","last_clean":"2026-02-21T14:59:51.817932+0000","last_became_active":"2026-02-21T14:58:47.134964+0000","last_became_peered":"2026-02-21T14:58:47.134964+0000","last_unstale":"2026-02-21T14:59:51.817932+0000","last_undegraded":"2026-02-21T14:59:51.817932+0000","last_fullsized":"2026-02-21T14:59:51.817932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'2","last_scrub_stamp":"2026-02-21T14:59:51.841842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:59:51.841842+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":6,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'2","reported_seq":35,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:51.816945+0000","last_change":"2026-02-21T14:59:51.844738+0000","last_active":"2026-02-21T14:59:51.816945+0000","last_peered":"2026-02-21T14:59:51.816945+0000","last_clean":"2026-02-21T14:59:51.816945+0000","last_became_active":"2026-02-21T14:58:47.138164+0000","last_became_peered":"2026-02-21T14:58:47.138164+0000","last_unstale":"2026-02-21T14:59:51.816945+0000","last_undegraded":"2026-02-21T14:59:51.816945+0000","last_fullsized":"2026-02-21T14:59:51.816945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'2","last_scrub_stamp":"2026-02-21T14:59:51.844638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:59:51.844638+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":44,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:51.510633+0000","last_change":"2026-02-21T14:59:51.527912+0000","last_active":"2026-02-21T14:59:51.510633+0000","last_peered":"2026-02-21T14:59:51.510633+0000","last_clean":"2026-02-21T14:59:51.510633+0000","last_became_active":"2026-02-21T14:58:47.153243+0000","last_became_peered":"2026-02-21T14:58:47.153243+0000","last_unstale":"2026-02-21T14:59:51.510633+0000","last_undegraded":"2026-02-21T14:59:51.510633+0000","last_fullsized":"2026-02-21T14:59:51.510633+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"17'2","last_scrub_stamp":"2026-02-21T14:59:51.527846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:59:51.527846+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'1","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:52.121632+0000","last_change":"2026-02-21T14:59:52.133372+0000","last_active":"2026-02-21T14:59:52.121632+0000","last_peered":"2026-02-21T14:59:52.121632+0000","last_clean":"2026-02-21T14:59:52.121632+0000","last_became_active":"2026-02-21T14:58:47.146202+0000","last_became_peered":"2026-02-21T14:58:47.146202+0000","last_unstale":"2026-02-21T14:59:52.121632+0000","last_undegraded":"2026-02-21T14:59:52.121632+0000","last_fullsized":"2026-02-21T14:59:52.121632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2026-02-21T14:59:52.133285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:59:52.133285+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"21'210","reported_seq":447,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:52.120671+0000","last_change":"2026-02-21T14:59:52.133398+0000","last_active":"2026-02-21T14:59:52.120671+0000","last_peered":"2026-02-21T14:59:52.120671+0000","last_clean":"2026-02-21T14:59:52.120671+0000","last_became_active":"2026-02-21T14:58:47.160042+0000","last_became_peered":"2026-02-21T14:58:47.160042+0000","last_unstale":"2026-02-21T14:59:52.120671+0000","last_undegraded":"2026-02-21T14:59:52.120671+0000","last_fullsized":"2026-02-21T14:59:52.120671+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'210","last_scrub_stamp":"2026-02-21T14:59:52.133345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:59:52.133345+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":832,"num_write":210,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":38,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:51.816132+0000","last_change":"2026-02-21T14:59:51.839639+0000","last_active":"2026-02-21T14:59:51.816132+0000","last_peered":"2026-02-21T14:59:51.816132+0000","last_clean":"2026-02-21T14:59:51.816132+0000","last_became_active":"2026-02-21T14:58:47.124462+0000","last_became_peered":"2026-02-21T14:58:47.124462+0000","last_unstale":"2026-02-21T14:59:51.816132+0000","last_undegraded":"2026-02-21T14:59:51.816132+0000","last_fullsized":"2026-02-21T14:59:51.816132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'6","last_scrub_stamp":"2026-02-21T14:59:51.839566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:59:51.839566+0000","last_clean_scrub_stamp":"2026-02-21T14:58:46.054218+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":104,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-21T14:59:51.815434+0000","last_change":"2026-02-21T14:59:51.872635+0000","last_active":"2026-02-21T14:59:51.815434+0000","last_peered":"2026-02-21T14:59:51.815434+0000","last_clean":"2026-02-21T14:59:51.815434+0000","last_became_active":"2026-02-21T14:58:44.090205+0000","last_became_peered":"2026-02-21T14:58:44.090205+0000","last_unstale":"2026-02-21T14:59:51.815434+0000","last_undegraded":"2026-02-21T14:59:51.815434+0000","last_fullsized":"2026-02-21T14:59:51.815434+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'52","last_scrub_stamp":"2026-02-21T14:59:51.872561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T14:59:51.872561+0000","last_clean_scrub_stamp":"2026-02-21T14:58:43.041461+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":477,"num_read_kb":879,"num_write":222,"num_write_kb":844,"num_scrub_errors":0,"num_shallow_scrub_errors":2,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160252743680,"internally_reserved":0,"allocated":3223552,"data_stored":3223552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417209856,"internally_reserved":0,"allocated":1445888,"data_stored":1445888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949673049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":540,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166116,"statfs":{"total":53418655744,"available":53418102784,"internally_reserved":0,"allocated":552960,"data_stored":552960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1196,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165460,"statfs":{"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1412,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165244,"statfs":{"total":53418655744,"available":53417209856,"internally_reserved":0,"allocated":1445888,"data_stored":1445888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417209856,"internally_reserved":0,"allocated":1445888,"data_stored":1445888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417209856,"internally_reserved":0,"allocated":1445888,"data_stored":1445888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418102784,"internally_reserved":0,"allocated":552960,"data_stored":552960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T15:00:12.540 DEBUG:teuthology.orchestra.run.trial130:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-21T15:00:12.616 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T15:00:12.621+0000 7f15428b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T15:00:12.618 INFO:teuthology.orchestra.run.trial130.stderr:2026-02-21T15:00:12.627+0000 7f15428b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T15:00:12.749 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-21T15:00:12.750 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-21T15:00:12.750 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-21T15:00:12.750 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T15:00:12.813 INFO:tasks.ceph.osd.0:Stopped 2026-02-21T15:00:12.813 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-21T15:00:12.813 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T15:00:12.876 INFO:tasks.ceph.osd.1:Stopped 2026-02-21T15:00:12.876 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-21T15:00:12.877 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T15:00:12.945 INFO:tasks.ceph.osd.2:Stopped 2026-02-21T15:00:12.945 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-21T15:00:12.946 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-21T15:00:12.946 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T15:00:12.982 INFO:tasks.ceph.mgr.x:Stopped 2026-02-21T15:00:12.982 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-21T15:00:12.982 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-21T15:00:12.982 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T15:00:13.032 INFO:tasks.ceph.mon.a:Stopped 2026-02-21T15:00:13.032 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-21T15:00:13.032 DEBUG:teuthology.orchestra.run.trial130:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2026-02-21T15:00:13.098 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial130.front.sepia.ceph.com 2026-02-21T15:00:13.098 DEBUG:teuthology.orchestra.run.trial130:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-21T15:00:13.166 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial130.front.sepia.ceph.com 2026-02-21T15:00:13.166 DEBUG:teuthology.orchestra.run.trial130:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-21T15:00:13.233 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial130.front.sepia.ceph.com 2026-02-21T15:00:13.233 DEBUG:teuthology.orchestra.run.trial130:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-21T15:00:13.306 INFO:tasks.ceph:Archiving mon data... 2026-02-21T15:00:13.308 DEBUG:teuthology.misc:Transferring archived files from trial130:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/xuxuehan-2026-02-21_14:23:14-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62724/data/mon.a.tgz 2026-02-21T15:00:13.308 DEBUG:teuthology.orchestra.run.trial130:> mktemp 2026-02-21T15:00:13.363 INFO:teuthology.orchestra.run.trial130.stdout:/tmp/tmp.AVzbRcRZj0 2026-02-21T15:00:13.363 DEBUG:teuthology.orchestra.run.trial130:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.AVzbRcRZj0 2026-02-21T15:00:13.449 DEBUG:teuthology.orchestra.run.trial130:> sudo chmod 0666 /tmp/tmp.AVzbRcRZj0 2026-02-21T15:00:13.531 DEBUG:teuthology.orchestra.remote:trial130:/tmp/tmp.AVzbRcRZj0 is 178KB 2026-02-21T15:00:13.582 DEBUG:teuthology.orchestra.run.trial130:> rm -fr /tmp/tmp.AVzbRcRZj0 2026-02-21T15:00:13.598 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-21T15:00:13.598 DEBUG:teuthology.orchestra.run.trial130:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-21T15:00:13.718 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-21T15:00:13.718 INFO:tasks.ceph:Archiving crash dumps... 2026-02-21T15:00:13.720 DEBUG:teuthology.misc:Transferring archived files from trial130:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/xuxuehan-2026-02-21_14:23:14-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62724/remote/trial130/crash 2026-02-21T15:00:13.721 DEBUG:teuthology.orchestra.run.trial130:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-21T15:00:13.781 INFO:tasks.ceph:Compressing logs... 2026-02-21T15:00:13.782 DEBUG:teuthology.orchestra.run.trial130:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T15:00:13.845 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-21T15:00:13.861 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-21T15:00:13.861 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42075.log 2026-02-21T15:00:13.861 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40980.log 2026-02-21T15:00:13.861 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42418.log 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42075.log.gz 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40980.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40980.log.gz 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41838.log 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42418.log.gz 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41100.log 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39813.log 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.41838.log.gz 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40730.log 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41100.log.gz 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.39813.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38426.log 2026-02-21T15:00:13.862 INFO:teuthology.orchestra.run.trial130.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39813.log.gz 2026-02-21T15:00:13.863 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40091.log 2026-02-21T15:00:13.863 INFO:teuthology.orchestra.run.trial130.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40730.log.gz 2026-02-21T15:00:13.863 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph.tmp-client.admin.38426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41274.log 2026-02-21T15:00:13.863 INFO:teuthology.orchestra.run.trial130.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38426.log.gz 2026-02-21T15:00:13.863 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42516.log 2026-02-21T15:00:13.863 INFO:teuthology.orchestra.run.trial130.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40091.log.gz 2026-02-21T15:00:13.863 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41274.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41274.log.gz 2026-02-21T15:00:13.863 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41220.log 2026-02-21T15:00:13.863 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42565.log 2026-02-21T15:00:13.863 INFO:teuthology.orchestra.run.trial130.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42516.log.gz 2026-02-21T15:00:13.863 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41930.log 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41220.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41220.log.gz 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42565.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42565.log.gz 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42222.log 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41367.log 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.41930.log.gz 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42222.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42222.log.gz 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41099.log 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42467.log 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41367.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41367.log.gz 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41099.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41099.log.gz 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40665.log 2026-02-21T15:00:13.864 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41909.log 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.42467.log.gz 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40665.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.40665.log.gz 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41753.log 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41465.log 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.41909.log.gz 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41753.log: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.41753.log.gz 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42173.log 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41465.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41465.log.gz 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41275.log 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42173.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42173.log.gz 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39860.log 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41275.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41275.log.gz 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42615.log 2026-02-21T15:00:13.865 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.39860.log: 48.8% -- replaced with /var/log/ceph/ceph-client.admin.39860.log.gz 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42369.log 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40640.log 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42615.log.gz 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42369.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42369.log.gz 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42124.log 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40570.log 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40640.log.gz 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42124.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42124.log.gz 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41416.log 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39718.log 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40570.log.gz 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41416.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41416.log.gz 2026-02-21T15:00:13.866 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40493.log 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.39718.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39718.log.gz 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40493.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40493.log.gz 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40978.log 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40979.log 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40978.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40978.log.gz 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr: 86.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41735.log 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42271.log 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40979.log.gz 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40044.log 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.41735.log.gz 2026-02-21T15:00:13.867 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42271.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42271.log.gz 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41815.log 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40044.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40044.log.gz 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41815.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.41815.log.gz 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42320.log 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39997.log 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42320.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42320.log.gz 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40779.log 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.39997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41752.log 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39997.log.gz 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40779.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40779.log.gz 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41129.log 2026-02-21T15:00:13.868 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41752.log: 65.0% -- replaced with /var/log/ceph/ceph-client.admin.41752.log.gz 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41129.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41129.log.gz 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41793.log 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41885.log 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41793.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.41793.log.gz 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42662.log 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41885.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41885.log.gz 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.42662.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.42662.log.gz 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40828.log 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40877.log 2026-02-21T15:00:13.869 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40828.log: 84.1% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-21T15:00:13.870 INFO:teuthology.orchestra.run.trial130.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40828.log.gz 2026-02-21T15:00:13.870 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41860.log 2026-02-21T15:00:13.870 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40310.log 2026-02-21T15:00:13.870 INFO:teuthology.orchestra.run.trial130.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40877.log.gz 2026-02-21T15:00:13.870 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.41860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40341.log 2026-02-21T15:00:13.870 INFO:teuthology.orchestra.run.trial130.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41860.log.gz 2026-02-21T15:00:13.870 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40310.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40310.log.gz 2026-02-21T15:00:13.870 INFO:teuthology.orchestra.run.trial130.stderr:/var/log/ceph/ceph-client.admin.40341.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40341.log.gz 2026-02-21T15:00:13.871 INFO:teuthology.orchestra.run.trial130.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-21T15:00:13.874 INFO:teuthology.orchestra.run.trial130.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-21T15:00:14.251 INFO:teuthology.orchestra.run.trial130.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-21T15:00:14.417 INFO:teuthology.orchestra.run.trial130.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-21T15:00:14.542 INFO:teuthology.orchestra.run.trial130.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-21T15:00:14.544 INFO:teuthology.orchestra.run.trial130.stderr: 2026-02-21T15:00:14.544 INFO:teuthology.orchestra.run.trial130.stderr:real 0m0.707s 2026-02-21T15:00:14.544 INFO:teuthology.orchestra.run.trial130.stderr:user 0m1.628s 2026-02-21T15:00:14.544 INFO:teuthology.orchestra.run.trial130.stderr:sys 0m0.122s 2026-02-21T15:00:14.545 INFO:tasks.ceph:Archiving logs... 2026-02-21T15:00:14.545 DEBUG:teuthology.misc:Transferring archived files from trial130:/var/log/ceph to /home/teuthworker/mnt/teuthology/xuxuehan-2026-02-21_14:23:14-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62724/remote/trial130/log 2026-02-21T15:00:14.546 DEBUG:teuthology.orchestra.run.trial130:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-21T15:00:14.915 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T15:00:14.984 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T15:00:14.996 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-21T15:00:14.996 DEBUG:teuthology.orchestra.run.trial130:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T15:00:15.021 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-21T15:00:15.032 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-21T15:00:15.033 DEBUG:teuthology.orchestra.run.trial130:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T15:00:15.076 INFO:teuthology.orchestra.run.trial130.stderr:bash: line 1: ntpq: command not found 2026-02-21T15:00:15.094 INFO:teuthology.orchestra.run.trial130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-21T15:00:15.094 INFO:teuthology.orchestra.run.trial130.stdout:=============================================================================== 2026-02-21T15:00:15.094 INFO:teuthology.orchestra.run.trial130.stdout:^- 172-233-177-198.ip.linod> 2 7 377 41 -3788us[-3788us] +/- 85ms 2026-02-21T15:00:15.094 INFO:teuthology.orchestra.run.trial130.stdout:^- chi3.us.ntp.li 2 6 377 52 -52us[ -52us] +/- 11ms 2026-02-21T15:00:15.094 INFO:teuthology.orchestra.run.trial130.stdout:^- ntp2.wiktel.com 1 8 377 113 -1231us[-1231us] +/- 20ms 2026-02-21T15:00:15.094 INFO:teuthology.orchestra.run.trial130.stdout:^* pool-138-89-14-60.mad.ea> 1 8 377 119 -128us[ -146us] +/- 3135us 2026-02-21T15:00:15.094 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-21T15:00:15.106 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-21T15:00:15.106 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-21T15:00:15.122 DEBUG:teuthology.orchestra.run.trial130:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-21T15:00:15.189 DEBUG:teuthology.orchestra.run.trial130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-21T15:00:15.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T15:00:15.216 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-21T15:00:15.232 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-21T15:00:15.247 INFO:teuthology.task.internal:Duration was 1880.799398 seconds 2026-02-21T15:00:15.247 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-21T15:00:15.263 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-21T15:00:15.263 DEBUG:teuthology.orchestra.run.trial130:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T15:00:15.300 INFO:teuthology.orchestra.run.trial130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-21T15:00:15.691 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-21T15:00:15.691 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial130.front.sepia.ceph.com 2026-02-21T15:00:15.691 DEBUG:teuthology.orchestra.run.trial130:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T15:00:15.749 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-21T15:00:15.750 DEBUG:teuthology.orchestra.run.trial130:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T15:00:15.850 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-21T15:00:15.850 DEBUG:teuthology.orchestra.run.trial130:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T15:00:15.874 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T15:00:15.875 INFO:teuthology.orchestra.run.trial130.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T15:00:15.875 INFO:teuthology.orchestra.run.trial130.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T15:00:15.875 INFO:teuthology.orchestra.run.trial130.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-21T15:00:15.875 INFO:teuthology.orchestra.run.trial130.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T15:00:15.884 INFO:teuthology.orchestra.run.trial130.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T15:00:15.887 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-21T15:00:15.906 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-21T15:00:15.906 DEBUG:teuthology.orchestra.run.trial130:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T15:00:15.951 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-21T15:00:15.968 DEBUG:teuthology.orchestra.run.trial130:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T15:00:16.014 INFO:teuthology.orchestra.run.trial130.stdout:kernel.core_pattern = core 2026-02-21T15:00:16.030 DEBUG:teuthology.orchestra.run.trial130:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T15:00:16.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T15:00:16.086 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-21T15:00:16.103 INFO:teuthology.task.internal:Transferring archived files... 2026-02-21T15:00:16.104 DEBUG:teuthology.misc:Transferring archived files from trial130:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/xuxuehan-2026-02-21_14:23:14-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62724/remote/trial130 2026-02-21T15:00:16.104 DEBUG:teuthology.orchestra.run.trial130:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T15:00:16.161 INFO:teuthology.task.internal:Removing archive directory... 2026-02-21T15:00:16.161 DEBUG:teuthology.orchestra.run.trial130:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T15:00:16.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-21T15:00:16.228 INFO:teuthology.task.internal:Not uploading archives. 2026-02-21T15:00:16.228 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-21T15:00:16.244 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-21T15:00:16.245 DEBUG:teuthology.orchestra.run.trial130:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T15:00:16.266 INFO:teuthology.orchestra.run.trial130.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 21 15:00 /home/ubuntu/cephtest 2026-02-21T15:00:16.267 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-21T15:00:16.285 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-21T15:00:16.325 INFO:teuthology.run:Summary data: description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} tasks/rbd_lock_and_fence} duration: 1880.7993977069855 flavor: debug owner: scheduled_xuxuehan@soko04.front.sepia.ceph.com success: true 2026-02-21T15:00:16.326 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T15:00:16.440 INFO:teuthology.run:pass