2026-02-20T11:31:19.023 INFO:root:teuthology version: 1.2.4.dev11+ge6d9099ce 2026-02-20T11:31:19.039 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T11:31:19.118 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60546 branch: wip-perezjos-crimson-only-19-02-2026-PR64439 description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rbd_lock_and_fence} email: null first_in_suite: false flavor: debug job_id: '60546' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-perezjos-crimson-only-19-02-2026-PR64439 ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 445d5c9ba18a4040e70e25303230c36c77773acb owner: scheduled_jjperez@soko04.front.sepia.ceph.com priority: 101 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 1736 sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 445d5c9ba18a4040e70e25303230c36c77773acb targets: trial099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLeO7dFB75VWN+Jk4X2HfPPHd0/rSpRilN07tGug2dA8CXWfWX6dhjWKP5rpaawg1/5dPbSRnMOidpRXtE2FFTw= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 - workunit: clients: client.0: - rbd/test_lock_fence.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: reimage-ssh-down teuthology_sha1: e6d9099ce03dc21193fda653159ec82465c3a94a timestamp: 2026-02-20_10:21:31 tube: trial user: jjperez verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-20T11:31:19.119 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa; will attempt to use it 2026-02-20T11:31:19.119 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks 2026-02-20T11:31:19.119 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-20T11:31:19.121 INFO:teuthology.task.internal:Checking packages... 2026-02-20T11:31:19.121 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4' 2026-02-20T11:31:19.121 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-20T11:31:19.121 INFO:teuthology.packaging:ref: None 2026-02-20T11:31:19.121 INFO:teuthology.packaging:tag: None 2026-02-20T11:31:19.121 INFO:teuthology.packaging:branch: wip-perezjos-crimson-only-19-02-2026-PR64439 2026-02-20T11:31:19.121 INFO:teuthology.packaging:sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:31:19.121 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-perezjos-crimson-only-19-02-2026-PR64439 2026-02-20T11:31:19.237 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5357.g0c3148ca 2026-02-20T11:31:19.238 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-20T11:31:19.250 INFO:teuthology.task.internal:no buildpackages task found 2026-02-20T11:31:19.250 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-20T11:31:19.263 INFO:teuthology.task.internal:Saving configuration 2026-02-20T11:31:19.273 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-20T11:31:19.285 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-20T11:31:19.359 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial099.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60546', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-20 11:27:55.576249', 'locked_by': 'scheduled_jjperez@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLeO7dFB75VWN+Jk4X2HfPPHd0/rSpRilN07tGug2dA8CXWfWX6dhjWKP5rpaawg1/5dPbSRnMOidpRXtE2FFTw='} 2026-02-20T11:31:19.359 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-20T11:31:19.372 INFO:teuthology.task.internal:roles: ubuntu@trial099.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-20T11:31:19.372 INFO:teuthology.run_tasks:Running task console_log... 2026-02-20T11:31:19.454 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x73eaca313240>, signals=[15]) 2026-02-20T11:31:19.454 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-20T11:31:19.465 INFO:teuthology.task.internal:Opening connections... 2026-02-20T11:31:19.465 DEBUG:teuthology.task.internal:connecting to ubuntu@trial099.front.sepia.ceph.com 2026-02-20T11:31:19.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T11:31:19.526 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-20T11:31:19.537 DEBUG:teuthology.orchestra.run.trial099:> uname -m 2026-02-20T11:31:19.601 INFO:teuthology.orchestra.run.trial099.stdout:x86_64 2026-02-20T11:31:19.601 DEBUG:teuthology.orchestra.run.trial099:> cat /etc/os-release 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:NAME="CentOS Stream" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:VERSION="9" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:ID="centos" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:ID_LIKE="rhel fedora" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:VERSION_ID="9" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:PLATFORM_ID="platform:el9" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:ANSI_COLOR="0;31" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:LOGO="fedora-logo-icon" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:HOME_URL="https://centos.org/" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-20T11:31:19.658 INFO:teuthology.orchestra.run.trial099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-20T11:31:19.659 INFO:teuthology.lock.ops:Updating trial099.front.sepia.ceph.com on lock server 2026-02-20T11:31:19.724 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-20T11:31:19.736 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-20T11:31:19.746 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-20T11:31:19.746 DEBUG:teuthology.orchestra.run.trial099:> test '!' -e /home/ubuntu/cephtest 2026-02-20T11:31:19.761 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-20T11:31:19.772 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-20T11:31:19.772 DEBUG:teuthology.orchestra.run.trial099:> test -z $(ls -A /var/lib/ceph) 2026-02-20T11:31:19.818 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-20T11:31:19.906 INFO:teuthology.run_tasks:Running task kernel... 2026-02-20T11:31:19.927 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-20T11:31:19.927 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-20T11:31:19.927 DEBUG:teuthology.orchestra.run.trial099:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-20T11:31:19.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:31:19.943 DEBUG:teuthology.orchestra.run.trial099:> uname -r 2026-02-20T11:31:19.999 INFO:teuthology.orchestra.run.trial099.stdout:5.14.0-681.el9.x86_64 2026-02-20T11:31:19.999 INFO:teuthology.task.kernel:Running kernel on trial099: 5.14.0-681.el9.x86_64 2026-02-20T11:31:19.999 DEBUG:teuthology.orchestra.run.trial099:> sudo yum install -y kernel 2026-02-20T11:31:21.335 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.9 MB 00:00 2026-02-20T11:31:24.559 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-20T11:31:28.294 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 7.8 MB 00:01 2026-02-20T11:31:29.237 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - Extras packages 94 kB/s | 20 kB 00:00 2026-02-20T11:31:29.818 INFO:teuthology.orchestra.run.trial099.stdout:Extra Packages for Enterprise Linux 39 MB/s | 20 MB 00:00 2026-02-20T11:31:33.067 INFO:teuthology.orchestra.run.trial099.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-20T11:31:34.078 INFO:teuthology.orchestra.run.trial099.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-20T11:31:34.078 INFO:teuthology.orchestra.run.trial099.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-20T11:31:34.097 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:31:34.098 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:31:34.098 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:31:34.122 DEBUG:teuthology.orchestra.run.trial099:> echo no | sudo yum reinstall kernel || true 2026-02-20T11:31:34.423 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 11:31:33 AM UTC. 2026-02-20T11:31:34.504 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:31:34.505 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:31:34.505 INFO:teuthology.orchestra.run.trial099.stdout: Package Architecture Version Repository Size 2026-02-20T11:31:34.506 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:31:34.506 INFO:teuthology.orchestra.run.trial099.stdout:Reinstalling: 2026-02-20T11:31:34.506 INFO:teuthology.orchestra.run.trial099.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T11:31:34.506 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:31:34.506 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:31:34.506 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:31:34.506 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:31:34.506 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 803 k 2026-02-20T11:31:34.506 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 0 2026-02-20T11:31:34.506 INFO:teuthology.orchestra.run.trial099.stderr:Operation aborted. 2026-02-20T11:31:34.526 INFO:teuthology.orchestra.run.trial099.stdout:Is this ok [y/N]: 2026-02-20T11:31:34.526 DEBUG:teuthology.orchestra.run.trial099:> sudo yum reinstall -y kernel || true 2026-02-20T11:31:34.825 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 11:31:33 AM UTC. 2026-02-20T11:31:34.904 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:31:34.905 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:31:34.905 INFO:teuthology.orchestra.run.trial099.stdout: Package Architecture Version Repository Size 2026-02-20T11:31:34.905 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:31:34.905 INFO:teuthology.orchestra.run.trial099.stdout:Reinstalling: 2026-02-20T11:31:34.905 INFO:teuthology.orchestra.run.trial099.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T11:31:34.905 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:31:34.906 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:31:34.906 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:31:34.906 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:31:34.906 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 803 k 2026-02-20T11:31:34.906 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 0 2026-02-20T11:31:34.906 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:31:35.039 INFO:teuthology.orchestra.run.trial099.stdout:kernel-5.14.0-681.el9.x86_64.rpm 11 MB/s | 803 kB 00:00 2026-02-20T11:31:35.039 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:31:35.039 INFO:teuthology.orchestra.run.trial099.stdout:Total 5.8 MB/s | 803 kB 00:00 2026-02-20T11:31:35.049 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:31:35.082 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:31:35.082 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:31:35.086 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:31:35.087 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:31:35.110 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:31:35.115 INFO:teuthology.orchestra.run.trial099.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T11:31:35.115 INFO:teuthology.orchestra.run.trial099.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:31:35.223 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:31:35.224 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T11:31:35.281 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:31:35.281 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:31:35.281 INFO:teuthology.orchestra.run.trial099.stdout:Reinstalled: 2026-02-20T11:31:35.281 INFO:teuthology.orchestra.run.trial099.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-20T11:31:35.282 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:31:35.282 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:31:35.309 DEBUG:teuthology.orchestra.run.trial099:> rpm -q kernel | sort -rV | head -n 1 2026-02-20T11:31:35.333 INFO:teuthology.orchestra.run.trial099.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-20T11:31:35.334 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-20T11:31:35.334 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-20T11:31:35.334 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-20T11:31:35.334 DEBUG:teuthology.orchestra.run.trial099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-20T11:31:35.398 INFO:teuthology.orchestra.run.trial099.stdout:ttyS1 2026-02-20T11:31:35.423 DEBUG:teuthology.parallel:result is None 2026-02-20T11:31:35.423 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-20T11:31:35.435 INFO:teuthology.task.internal:Creating test directory... 2026-02-20T11:31:35.435 DEBUG:teuthology.orchestra.run.trial099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-20T11:31:35.479 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-20T11:31:35.491 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-20T11:31:35.505 INFO:teuthology.task.internal:Creating archive directory... 2026-02-20T11:31:35.505 DEBUG:teuthology.orchestra.run.trial099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-20T11:31:35.540 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-20T11:31:35.552 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-20T11:31:35.552 DEBUG:teuthology.orchestra.run.trial099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-20T11:31:35.600 INFO:teuthology.orchestra.run.trial099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T11:31:35.609 INFO:teuthology.orchestra.run.trial099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T11:31:35.610 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-20T11:31:35.623 INFO:teuthology.task.internal:Configuring sudo... 2026-02-20T11:31:35.624 DEBUG:teuthology.orchestra.run.trial099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-20T11:31:35.675 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-20T11:31:35.691 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-20T11:31:35.691 DEBUG:teuthology.orchestra.run.trial099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-20T11:31:35.731 DEBUG:teuthology.orchestra.run.trial099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T11:31:35.906 DEBUG:teuthology.orchestra.run.trial099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T11:31:35.929 DEBUG:teuthology.orchestra.run.trial099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T11:31:36.065 DEBUG:teuthology.orchestra.run.trial099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T11:31:36.088 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:31:36.088 DEBUG:teuthology.orchestra.run.trial099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-20T11:31:36.152 DEBUG:teuthology.orchestra.run.trial099:> sudo service rsyslog restart 2026-02-20T11:31:36.220 INFO:teuthology.orchestra.run.trial099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T11:31:36.529 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-20T11:31:36.541 INFO:teuthology.task.internal:Starting timer... 2026-02-20T11:31:36.541 INFO:teuthology.run_tasks:Running task pcp... 2026-02-20T11:31:36.562 INFO:teuthology.run_tasks:Running task selinux... 2026-02-20T11:31:36.581 DEBUG:teuthology.orchestra.run.trial099:> sudo service auditd rotate 2026-02-20T11:31:36.609 INFO:teuthology.orchestra.run.trial099.stdout:Rotating logs: 2026-02-20T11:31:36.611 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-20T11:31:36.611 DEBUG:teuthology.orchestra.run.trial099:> /usr/sbin/getenforce 2026-02-20T11:31:36.667 INFO:teuthology.orchestra.run.trial099.stdout:Permissive 2026-02-20T11:31:36.667 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial099.front.sepia.ceph.com': 'permissive'} 2026-02-20T11:31:36.667 DEBUG:teuthology.orchestra.run.trial099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T11:31:36.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:31:36.732 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-20T11:31:36.732 DEBUG:teuthology.orchestra.run.trial099:> sudo /usr/sbin/setenforce permissive 2026-02-20T11:31:36.795 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-20T11:31:36.810 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-20T11:31:36.822 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-20T11:31:36.824 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial099.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-20T11:34:22.597 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial099.front.sepia.ceph.com')] 2026-02-20T11:34:22.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial099.front.sepia.ceph.com' 2026-02-20T11:34:22.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T11:34:22.664 DEBUG:teuthology.orchestra.run.trial099:> true 2026-02-20T11:34:22.738 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial099.front.sepia.ceph.com' 2026-02-20T11:34:22.739 INFO:teuthology.run_tasks:Running task clock... 2026-02-20T11:34:22.750 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-20T11:34:22.751 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-20T11:34:22.751 DEBUG:teuthology.orchestra.run.trial099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T11:34:22.806 INFO:teuthology.orchestra.run.trial099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-20T11:34:22.818 INFO:teuthology.orchestra.run.trial099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-20T11:34:22.837 INFO:teuthology.orchestra.run.trial099.stderr:sudo: ntpd: command not found 2026-02-20T11:34:22.846 INFO:teuthology.orchestra.run.trial099.stdout:506 Cannot talk to daemon 2026-02-20T11:34:22.859 INFO:teuthology.orchestra.run.trial099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-20T11:34:22.870 INFO:teuthology.orchestra.run.trial099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-20T11:34:22.917 INFO:teuthology.orchestra.run.trial099.stderr:bash: line 1: ntpq: command not found 2026-02-20T11:34:22.919 INFO:teuthology.orchestra.run.trial099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T11:34:22.919 INFO:teuthology.orchestra.run.trial099.stdout:=============================================================================== 2026-02-20T11:34:22.920 INFO:teuthology.run_tasks:Running task install... 2026-02-20T11:34:22.934 DEBUG:teuthology.task.install:project ceph 2026-02-20T11:34:22.934 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}} 2026-02-20T11:34:22.934 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'} 2026-02-20T11:34:22.935 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T11:34:22.944 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T11:34:22.944 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-20T11:34:22.945 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'tag': None, 'wait_for_package': False} 2026-02-20T11:34:22.946 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:34:23.111 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/ 2026-02-20T11:34:23.111 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5357.g0c3148ca 2026-02-20T11:34:23.506 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T11:34:23.506 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:34:23.506 DEBUG:teuthology.orchestra.run.trial099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T11:34:23.533 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-20T11:34:23.533 DEBUG:teuthology.orchestra.run.trial099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T11:34:23.600 DEBUG:teuthology.orchestra.run.trial099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T11:34:23.673 DEBUG:teuthology.orchestra.run.trial099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T11:34:23.737 INFO:teuthology.orchestra.run.trial099.stdout:check_obsoletes = 1 2026-02-20T11:34:23.738 DEBUG:teuthology.orchestra.run.trial099:> sudo yum clean all 2026-02-20T11:34:23.898 INFO:teuthology.orchestra.run.trial099.stdout:41 files removed 2026-02-20T11:34:23.913 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-radosgw 2026-02-20T11:34:24.757 INFO:teuthology.orchestra.run.trial099.stdout:ceph packages for x86_64 134 kB/s | 94 kB 00:00 2026-02-20T11:34:25.375 INFO:teuthology.orchestra.run.trial099.stdout:ceph noarch packages 41 kB/s | 24 kB 00:00 2026-02-20T11:34:25.976 INFO:teuthology.orchestra.run.trial099.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-20T11:34:30.823 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.9 MB 00:04 2026-02-20T11:34:40.341 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - AppStream 3.0 MB/s | 27 MB 00:08 2026-02-20T11:34:44.755 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 7.8 MB 00:02 2026-02-20T11:34:46.661 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 20 kB 00:01 2026-02-20T11:34:46.934 INFO:teuthology.orchestra.run.trial099.stdout:Extra Packages for Enterprise Linux 96 MB/s | 20 MB 00:00 2026-02-20T11:34:50.145 INFO:teuthology.orchestra.run.trial099.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-20T11:34:51.354 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:34:51.355 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: Package Arch Version Repository Size 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout:Installing: 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: ceph-radosgw x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 11 M 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout:Upgrading: 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: librados2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 3.3 M 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: librbd1 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.7 M 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout:Installing dependencies: 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: ceph-base x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 5.1 M 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: ceph-common x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 21 M 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: ceph-selinux x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 25 k 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: libcephfs2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 860 k 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: libradosstriper1 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 263 k 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: librgw2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 5.6 M 2026-02-20T11:34:51.356 INFO:teuthology.orchestra.run.trial099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: python3-ceph-argparse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 45 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: python3-ceph-common x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 185 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: python3-cephfs x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 204 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: python3-rados x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 324 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: python3-rbd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 305 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: python3-rgw x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 100 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout:Installing weak dependencies: 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: libcephfs-daemon x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 37 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 26 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout:Install 35 Packages 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout:Upgrade 2 Packages 2026-02-20T11:34:51.357 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:34:51.358 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 61 M 2026-02-20T11:34:51.358 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:34:55.313 INFO:teuthology.orchestra.run.trial099.stdout:(1/37): ceph-base-20.3.0-5357.g0c3148ca.el9.x86 1.4 MB/s | 5.1 MB 00:03 2026-02-20T11:34:55.375 INFO:teuthology.orchestra.run.trial099.stdout:(2/37): ceph-selinux-20.3.0-5357.g0c3148ca.el9. 409 kB/s | 25 kB 00:00 2026-02-20T11:34:55.438 INFO:teuthology.orchestra.run.trial099.stdout:(3/37): libcephfs-daemon-20.3.0-5357.g0c3148ca. 589 kB/s | 37 kB 00:00 2026-02-20T11:34:55.500 INFO:teuthology.orchestra.run.trial099.stdout:(4/37): libcephfs-proxy2-20.3.0-5357.g0c3148ca. 408 kB/s | 26 kB 00:00 2026-02-20T11:34:56.092 INFO:teuthology.orchestra.run.trial099.stdout:(5/37): libcephfs2-20.3.0-5357.g0c3148ca.el9.x8 1.4 MB/s | 860 kB 00:00 2026-02-20T11:34:56.789 INFO:teuthology.orchestra.run.trial099.stdout:(6/37): libradosstriper1-20.3.0-5357.g0c3148ca. 377 kB/s | 263 kB 00:00 2026-02-20T11:34:57.180 INFO:teuthology.orchestra.run.trial099.stdout:(7/37): ceph-radosgw-20.3.0-5357.g0c3148ca.el9. 2.0 MB/s | 11 MB 00:05 2026-02-20T11:34:57.233 INFO:teuthology.orchestra.run.trial099.stdout:(8/37): python3-ceph-argparse-20.3.0-5357.g0c31 851 kB/s | 45 kB 00:00 2026-02-20T11:34:57.340 INFO:teuthology.orchestra.run.trial099.stdout:(9/37): python3-ceph-common-20.3.0-5357.g0c3148 1.7 MB/s | 185 kB 00:00 2026-02-20T11:34:57.450 INFO:teuthology.orchestra.run.trial099.stdout:(10/37): python3-cephfs-20.3.0-5357.g0c3148ca.e 1.8 MB/s | 204 kB 00:00 2026-02-20T11:34:57.673 INFO:teuthology.orchestra.run.trial099.stdout:(11/37): python3-rados-20.3.0-5357.g0c3148ca.el 1.4 MB/s | 324 kB 00:00 2026-02-20T11:34:57.847 INFO:teuthology.orchestra.run.trial099.stdout:(12/37): python3-rbd-20.3.0-5357.g0c3148ca.el9. 1.7 MB/s | 305 kB 00:00 2026-02-20T11:34:57.907 INFO:teuthology.orchestra.run.trial099.stdout:(13/37): python3-rgw-20.3.0-5357.g0c3148ca.el9. 1.6 MB/s | 100 kB 00:00 2026-02-20T11:34:58.390 INFO:teuthology.orchestra.run.trial099.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 69 kB/s | 33 kB 00:00 2026-02-20T11:34:59.579 INFO:teuthology.orchestra.run.trial099.stdout:(15/37): librgw2-20.3.0-5357.g0c3148ca.el9.x86_ 2.0 MB/s | 5.6 MB 00:02 2026-02-20T11:34:59.894 INFO:teuthology.orchestra.run.trial099.stdout:(16/37): nvme-cli-2.16-1.el9.x86_64.rpm 784 kB/s | 1.2 MB 00:01 2026-02-20T11:35:00.212 INFO:teuthology.orchestra.run.trial099.stdout:(17/37): pciutils-3.7.0-7.el9.x86_64.rpm 147 kB/s | 93 kB 00:00 2026-02-20T11:35:00.276 INFO:teuthology.orchestra.run.trial099.stdout:(18/37): boost-program-options-1.75.0-13.el9.x8 273 kB/s | 104 kB 00:00 2026-02-20T11:35:00.409 INFO:teuthology.orchestra.run.trial099.stdout:(19/37): libnbd-1.20.3-4.el9.x86_64.rpm 829 kB/s | 164 kB 00:00 2026-02-20T11:35:00.458 INFO:teuthology.orchestra.run.trial099.stdout:(20/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 878 kB/s | 160 kB 00:00 2026-02-20T11:35:00.522 INFO:teuthology.orchestra.run.trial099.stdout:(21/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 401 kB/s | 45 kB 00:00 2026-02-20T11:35:00.680 INFO:teuthology.orchestra.run.trial099.stdout:(22/37): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2026-02-20T11:35:00.711 INFO:teuthology.orchestra.run.trial099.stdout:(23/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2026-02-20T11:35:00.789 INFO:teuthology.orchestra.run.trial099.stdout:(24/37): qatlib-25.08.0-2.el9.x86_64.rpm 2.1 MB/s | 240 kB 00:00 2026-02-20T11:35:00.840 INFO:teuthology.orchestra.run.trial099.stdout:(25/37): qatlib-service-25.08.0-2.el9.x86_64.rp 287 kB/s | 37 kB 00:00 2026-02-20T11:35:00.905 INFO:teuthology.orchestra.run.trial099.stdout:(26/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 571 kB/s | 66 kB 00:00 2026-02-20T11:35:01.106 INFO:teuthology.orchestra.run.trial099.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.1 MB/s | 308 kB 00:00 2026-02-20T11:35:01.121 INFO:teuthology.orchestra.run.trial099.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.6 MB/s | 25 kB 00:00 2026-02-20T11:35:01.132 INFO:teuthology.orchestra.run.trial099.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 4.5 MB/s | 49 kB 00:00 2026-02-20T11:35:01.139 INFO:teuthology.orchestra.run.trial099.stdout:(30/37): libarrow-9.0.0-15.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2026-02-20T11:35:01.143 INFO:teuthology.orchestra.run.trial099.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 6.1 MB/s | 67 kB 00:00 2026-02-20T11:35:01.152 INFO:teuthology.orchestra.run.trial099.stdout:(32/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 60 MB/s | 838 kB 00:00 2026-02-20T11:35:01.158 INFO:teuthology.orchestra.run.trial099.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 13 MB/s | 191 kB 00:00 2026-02-20T11:35:01.168 INFO:teuthology.orchestra.run.trial099.stdout:(34/37): thrift-0.15.0-4.el9.x86_64.rpm 102 MB/s | 1.6 MB 00:00 2026-02-20T11:35:01.438 INFO:teuthology.orchestra.run.trial099.stdout:(35/37): ceph-common-20.3.0-5357.g0c3148ca.el9. 2.1 MB/s | 21 MB 00:09 2026-02-20T11:35:02.822 INFO:teuthology.orchestra.run.trial099.stdout:(36/37): librados2-20.3.0-5357.g0c3148ca.el9.x8 2.0 MB/s | 3.3 MB 00:01 2026-02-20T11:35:02.842 INFO:teuthology.orchestra.run.trial099.stdout:(37/37): librbd1-20.3.0-5357.g0c3148ca.el9.x86_ 1.6 MB/s | 2.7 MB 00:01 2026-02-20T11:35:02.845 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:35:02.846 INFO:teuthology.orchestra.run.trial099.stdout:Total 5.3 MB/s | 61 MB 00:11 2026-02-20T11:35:02.948 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:35:02.969 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:35:02.969 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:35:03.220 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:35:03.220 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:35:03.516 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:35:03.667 INFO:teuthology.orchestra.run.trial099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-20T11:35:03.671 INFO:teuthology.orchestra.run.trial099.stdout: Upgrading : librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:35:03.722 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:35:03.723 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:35:03.736 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:35:03.738 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_6 4/39 2026-02-20T11:35:03.738 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:35:03.765 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:35:03.767 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 6/39 2026-02-20T11:35:03.775 INFO:teuthology.orchestra.run.trial099.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-20T11:35:03.777 INFO:teuthology.orchestra.run.trial099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-20T11:35:03.780 INFO:teuthology.orchestra.run.trial099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-20T11:35:03.784 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-20T11:35:03.796 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9. 11/39 2026-02-20T11:35:03.809 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/39 2026-02-20T11:35:03.810 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 13/39 2026-02-20T11:35:03.831 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 13/39 2026-02-20T11:35:03.835 INFO:teuthology.orchestra.run.trial099.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-20T11:35:03.864 INFO:teuthology.orchestra.run.trial099.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-20T11:35:03.868 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-20T11:35:03.874 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-20T11:35:03.876 INFO:teuthology.orchestra.run.trial099.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-20T11:35:03.881 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-20T11:35:04.000 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-20T11:35:04.001 INFO:teuthology.orchestra.run.trial099.stdout: Upgrading : librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 21/39 2026-02-20T11:35:04.027 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 21/39 2026-02-20T11:35:04.030 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 22/39 2026-02-20T11:35:04.036 INFO:teuthology.orchestra.run.trial099.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-20T11:35:04.294 INFO:teuthology.orchestra.run.trial099.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-20T11:35:04.295 INFO:teuthology.orchestra.run.trial099.stdout: Installing : librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 25/39 2026-02-20T11:35:04.312 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 25/39 2026-02-20T11:35:04.316 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 26/39 2026-02-20T11:35:04.322 INFO:teuthology.orchestra.run.trial099.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-20T11:35:04.324 INFO:teuthology.orchestra.run.trial099.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-20T11:35:04.325 INFO:teuthology.orchestra.run.trial099.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-20T11:35:04.343 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-20T11:35:04.460 INFO:teuthology.orchestra.run.trial099.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-20T11:35:04.463 INFO:teuthology.orchestra.run.trial099.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-20T11:35:04.744 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-20T11:35:04.744 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-20T11:35:04.744 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:35:04.787 INFO:teuthology.orchestra.run.trial099.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-20T11:35:04.789 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x8 33/39 2026-02-20T11:35:05.780 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:35:05.782 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:35:06.057 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:35:06.060 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 35/39 2026-02-20T11:35:06.100 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 35/39 2026-02-20T11:35:06.100 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-20T11:35:06.100 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-20T11:35:06.100 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:35:06.104 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:35:10.678 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:35:10.678 INFO:teuthology.orchestra.run.trial099.stdout:skipping the directory /sys 2026-02-20T11:35:10.679 INFO:teuthology.orchestra.run.trial099.stdout:skipping the directory /proc 2026-02-20T11:35:10.679 INFO:teuthology.orchestra.run.trial099.stdout:skipping the directory /mnt 2026-02-20T11:35:10.679 INFO:teuthology.orchestra.run.trial099.stdout:skipping the directory /var/tmp 2026-02-20T11:35:10.679 INFO:teuthology.orchestra.run.trial099.stdout:skipping the directory /home 2026-02-20T11:35:10.679 INFO:teuthology.orchestra.run.trial099.stdout:skipping the directory /root 2026-02-20T11:35:10.679 INFO:teuthology.orchestra.run.trial099.stdout:skipping the directory /tmp 2026-02-20T11:35:10.679 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:35:11.194 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 37/39 2026-02-20T11:35:11.218 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 37/39 2026-02-20T11:35:11.218 INFO:teuthology.orchestra.run.trial099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:35:11.218 INFO:teuthology.orchestra.run.trial099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-20T11:35:11.218 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T11:35:11.218 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T11:35:11.218 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:35:11.220 INFO:teuthology.orchestra.run.trial099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-20T11:35:11.233 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-20T11:35:11.233 INFO:teuthology.orchestra.run.trial099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:35:11.289 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:35:11.290 INFO:teuthology.orchestra.run.trial099.stdout:Creating group 'qat' with GID 992. 2026-02-20T11:35:11.290 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:35:11.596 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/39 2026-02-20T11:35:11.596 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:35:11.596 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:35:11.596 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 4/39 2026-02-20T11:35:11.596 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 6/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 7/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 8/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 9/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9. 10/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x8 11/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 13/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 14/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 15/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-20T11:35:11.597 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-20T11:35:11.598 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-20T11:35:11.599 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-20T11:35:11.599 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-20T11:35:11.599 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-20T11:35:11.599 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-20T11:35:11.599 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-20T11:35:11.599 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-20T11:35:11.599 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:35:11.599 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-20T11:35:11.599 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 38/39 2026-02-20T11:35:11.658 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:35:11.658 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:35:11.658 INFO:teuthology.orchestra.run.trial099.stdout:Upgraded: 2026-02-20T11:35:11.658 INFO:teuthology.orchestra.run.trial099.stdout: librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.658 INFO:teuthology.orchestra.run.trial099.stdout: librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout:Installed: 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout: ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout: ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout: ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout: libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout: libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.659 INFO:teuthology.orchestra.run.trial099.stdout: libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-20T11:35:11.660 INFO:teuthology.orchestra.run.trial099.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: re2-1:20211101-20.el9.x86_64 2026-02-20T11:35:11.661 INFO:teuthology.orchestra.run.trial099.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-20T11:35:11.662 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:35:11.662 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:35:11.700 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-test 2026-02-20T11:35:12.008 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:22 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:35:12.091 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:35:12.092 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:35:12.092 INFO:teuthology.orchestra.run.trial099.stdout: Package Arch Version Repository Size 2026-02-20T11:35:12.092 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:35:12.092 INFO:teuthology.orchestra.run.trial099.stdout:Installing: 2026-02-20T11:35:12.093 INFO:teuthology.orchestra.run.trial099.stdout: ceph-test x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 127 M 2026-02-20T11:35:12.093 INFO:teuthology.orchestra.run.trial099.stdout:Installing dependencies: 2026-02-20T11:35:12.093 INFO:teuthology.orchestra.run.trial099.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-20T11:35:12.093 INFO:teuthology.orchestra.run.trial099.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-20T11:35:12.093 INFO:teuthology.orchestra.run.trial099.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-20T11:35:12.093 INFO:teuthology.orchestra.run.trial099.stdout: libcephsqlite x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 153 k 2026-02-20T11:35:12.093 INFO:teuthology.orchestra.run.trial099.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-20T11:35:12.093 INFO:teuthology.orchestra.run.trial099.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-20T11:35:12.093 INFO:teuthology.orchestra.run.trial099.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-20T11:35:12.093 INFO:teuthology.orchestra.run.trial099.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-20T11:35:12.093 INFO:teuthology.orchestra.run.trial099.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-20T11:35:12.094 INFO:teuthology.orchestra.run.trial099.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-20T11:35:12.094 INFO:teuthology.orchestra.run.trial099.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-20T11:35:12.094 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:35:12.094 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:35:12.094 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:35:12.094 INFO:teuthology.orchestra.run.trial099.stdout:Install 12 Packages 2026-02-20T11:35:12.094 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:35:12.094 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 132 M 2026-02-20T11:35:12.094 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 989 M 2026-02-20T11:35:12.094 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:35:13.044 INFO:teuthology.orchestra.run.trial099.stdout:(1/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 6.8 MB/s | 2.1 MB 00:00 2026-02-20T11:35:13.073 INFO:teuthology.orchestra.run.trial099.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 795 kB/s | 23 kB 00:00 2026-02-20T11:35:13.111 INFO:teuthology.orchestra.run.trial099.stdout:(3/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 2.5 MB/s | 94 kB 00:00 2026-02-20T11:35:13.292 INFO:teuthology.orchestra.run.trial099.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 2.8 MB/s | 525 kB 00:00 2026-02-20T11:35:13.322 INFO:teuthology.orchestra.run.trial099.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 5.9 MB/s | 176 kB 00:00 2026-02-20T11:35:13.353 INFO:teuthology.orchestra.run.trial099.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 7.3 MB/s | 233 kB 00:00 2026-02-20T11:35:13.411 INFO:teuthology.orchestra.run.trial099.stdout:(7/12): protobuf-3.14.0-17.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2026-02-20T11:35:13.439 INFO:teuthology.orchestra.run.trial099.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 11 MB/s | 303 kB 00:00 2026-02-20T11:35:13.464 INFO:teuthology.orchestra.run.trial099.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.4 MB/s | 64 kB 00:00 2026-02-20T11:35:13.617 INFO:teuthology.orchestra.run.trial099.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 797 kB/s | 122 kB 00:00 2026-02-20T11:35:25.060 INFO:teuthology.orchestra.run.trial099.stdout:(11/12): libcephsqlite-20.3.0-5357.g0c3148ca.el 12 kB/s | 153 kB 00:12 2026-02-20T11:35:53.368 INFO:teuthology.orchestra.run.trial099.stdout:(12/12): ceph-test-20.3.0-5357.g0c3148ca.el9.x8 3.1 MB/s | 127 MB 00:40 2026-02-20T11:35:53.370 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:35:53.370 INFO:teuthology.orchestra.run.trial099.stdout:Total 3.2 MB/s | 132 MB 00:41 2026-02-20T11:35:53.444 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:35:53.455 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:35:53.455 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:35:53.885 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:35:53.886 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:35:54.342 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:35:54.350 INFO:teuthology.orchestra.run.trial099.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-20T11:35:54.359 INFO:teuthology.orchestra.run.trial099.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-20T11:35:54.364 INFO:teuthology.orchestra.run.trial099.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-20T11:35:54.368 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-20T11:35:54.370 INFO:teuthology.orchestra.run.trial099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-20T11:35:54.375 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-20T11:35:54.376 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T11:35:54.393 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T11:35:54.395 INFO:teuthology.orchestra.run.trial099.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-20T11:35:54.407 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-20T11:35:54.416 INFO:teuthology.orchestra.run.trial099.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-20T11:35:54.416 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 11/12 2026-02-20T11:36:01.373 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 11/12 2026-02-20T11:36:01.391 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/12 2026-02-20T11:36:02.184 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/12 2026-02-20T11:36:02.184 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/12 2026-02-20T11:36:02.184 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/12 2026-02-20T11:36:02.184 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-20T11:36:02.184 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-20T11:36:02.184 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-20T11:36:02.184 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-20T11:36:02.185 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-20T11:36:02.185 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-20T11:36:02.185 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-20T11:36:02.185 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-20T11:36:02.185 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout:Installed: 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-20T11:36:02.237 INFO:teuthology.orchestra.run.trial099.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-20T11:36:02.238 INFO:teuthology.orchestra.run.trial099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-20T11:36:02.238 INFO:teuthology.orchestra.run.trial099.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-20T11:36:02.238 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:36:02.238 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:36:02.278 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph 2026-02-20T11:36:02.580 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:01:12 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:36:02.666 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:36:02.669 INFO:teuthology.orchestra.run.trial099.stdout:======================================================================================= 2026-02-20T11:36:02.669 INFO:teuthology.orchestra.run.trial099.stdout: Package Arch Version Repository Size 2026-02-20T11:36:02.669 INFO:teuthology.orchestra.run.trial099.stdout:======================================================================================= 2026-02-20T11:36:02.669 INFO:teuthology.orchestra.run.trial099.stdout:Installing: 2026-02-20T11:36:02.669 INFO:teuthology.orchestra.run.trial099.stdout: ceph x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 6.5 k 2026-02-20T11:36:02.669 INFO:teuthology.orchestra.run.trial099.stdout:Installing dependencies: 2026-02-20T11:36:02.669 INFO:teuthology.orchestra.run.trial099.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-20T11:36:02.669 INFO:teuthology.orchestra.run.trial099.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 57 k 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mds x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.0 M 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 937 k 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 297 k 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mon x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 4.9 M 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: ceph-osd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 163 k 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: ceph-osd-classic x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 15 M 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: ceph-osd-crimson x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 88 M 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 17 k 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: cephadm noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 1.0 M 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-20T11:36:02.670 INFO:teuthology.orchestra.run.trial099.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-20T11:36:02.671 INFO:teuthology.orchestra.run.trial099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-20T11:36:02.671 INFO:teuthology.orchestra.run.trial099.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-20T11:36:02.671 INFO:teuthology.orchestra.run.trial099.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-20T11:36:02.671 INFO:teuthology.orchestra.run.trial099.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-20T11:36:02.671 INFO:teuthology.orchestra.run.trial099.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-20T11:36:02.671 INFO:teuthology.orchestra.run.trial099.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-20T11:36:02.671 INFO:teuthology.orchestra.run.trial099.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-20T11:36:02.671 INFO:teuthology.orchestra.run.trial099.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-20T11:36:02.671 INFO:teuthology.orchestra.run.trial099.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-20T11:36:02.671 INFO:teuthology.orchestra.run.trial099.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-20T11:36:02.671 INFO:teuthology.orchestra.run.trial099.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-20T11:36:02.672 INFO:teuthology.orchestra.run.trial099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-20T11:36:02.673 INFO:teuthology.orchestra.run.trial099.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-20T11:36:02.674 INFO:teuthology.orchestra.run.trial099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-20T11:36:02.675 INFO:teuthology.orchestra.run.trial099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-20T11:36:02.676 INFO:teuthology.orchestra.run.trial099.stdout:Installing weak dependencies: 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 184 k 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 51 M 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 7.4 M 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 22 k 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-rook noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 50 k 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout: ceph-volume noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 301 k 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout:======================================================================================= 2026-02-20T11:36:02.677 INFO:teuthology.orchestra.run.trial099.stdout:Install 96 Packages 2026-02-20T11:36:02.678 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:36:02.678 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 226 M 2026-02-20T11:36:02.678 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 1.5 G 2026-02-20T11:36:02.679 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:36:03.570 INFO:teuthology.orchestra.run.trial099.stdout:(1/96): ceph-20.3.0-5357.g0c3148ca.el9.x86_64.r 24 kB/s | 6.5 kB 00:00 2026-02-20T11:36:04.121 INFO:teuthology.orchestra.run.trial099.stdout:(2/96): ceph-mgr-20.3.0-5357.g0c3148ca.el9.x86_ 1.1 MB/s | 937 kB 00:00 2026-02-20T11:36:04.193 INFO:teuthology.orchestra.run.trial099.stdout:(3/96): ceph-osd-20.3.0-5357.g0c3148ca.el9.x86_ 2.2 MB/s | 163 kB 00:00 2026-02-20T11:36:04.323 INFO:teuthology.orchestra.run.trial099.stdout:(4/96): ceph-mds-20.3.0-5357.g0c3148ca.el9.x86_ 2.0 MB/s | 2.0 MB 00:01 2026-02-20T11:36:05.529 INFO:teuthology.orchestra.run.trial099.stdout:(5/96): ceph-mon-20.3.0-5357.g0c3148ca.el9.x86_ 2.5 MB/s | 4.9 MB 00:01 2026-02-20T11:36:05.594 INFO:teuthology.orchestra.run.trial099.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5357.g0c 880 kB/s | 57 kB 00:00 2026-02-20T11:36:05.663 INFO:teuthology.orchestra.run.trial099.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5357.g0c3148ca. 2.6 MB/s | 184 kB 00:00 2026-02-20T11:36:10.846 INFO:teuthology.orchestra.run.trial099.stdout:(8/96): ceph-osd-classic-20.3.0-5357.g0c3148ca. 2.2 MB/s | 15 MB 00:06 2026-02-20T11:36:14.762 INFO:teuthology.orchestra.run.trial099.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-53 1.9 MB/s | 7.4 MB 00:03 2026-02-20T11:36:14.827 INFO:teuthology.orchestra.run.trial099.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5357.g0c3148 341 kB/s | 22 kB 00:00 2026-02-20T11:36:15.080 INFO:teuthology.orchestra.run.trial099.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5357.g0c3 1.1 MB/s | 297 kB 00:00 2026-02-20T11:36:15.149 INFO:teuthology.orchestra.run.trial099.stdout:(12/96): ceph-mgr-rook-20.3.0-5357.g0c3148ca.el 727 kB/s | 50 kB 00:00 2026-02-20T11:36:15.197 INFO:teuthology.orchestra.run.trial099.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5357.g0c 361 kB/s | 17 kB 00:00 2026-02-20T11:36:15.418 INFO:teuthology.orchestra.run.trial099.stdout:(14/96): ceph-volume-20.3.0-5357.g0c3148ca.el9. 1.3 MB/s | 301 kB 00:00 2026-02-20T11:36:16.244 INFO:teuthology.orchestra.run.trial099.stdout:(15/96): cephadm-20.3.0-5357.g0c3148ca.el9.noar 1.2 MB/s | 1.0 MB 00:00 2026-02-20T11:36:17.161 INFO:teuthology.orchestra.run.trial099.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 383 kB/s | 351 kB 00:00 2026-02-20T11:36:17.269 INFO:teuthology.orchestra.run.trial099.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 376 kB/s | 40 kB 00:00 2026-02-20T11:36:17.463 INFO:teuthology.orchestra.run.trial099.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 371 kB/s | 72 kB 00:00 2026-02-20T11:36:18.529 INFO:teuthology.orchestra.run.trial099.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 745 kB/s | 794 kB 00:01 2026-02-20T11:36:18.812 INFO:teuthology.orchestra.run.trial099.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 651 kB/s | 184 kB 00:00 2026-02-20T11:36:19.183 INFO:teuthology.orchestra.run.trial099.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 684 kB/s | 253 kB 00:00 2026-02-20T11:36:20.861 INFO:teuthology.orchestra.run.trial099.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 760 kB/s | 1.2 MB 00:01 2026-02-20T11:36:21.056 INFO:teuthology.orchestra.run.trial099.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 545 kB/s | 106 kB 00:00 2026-02-20T11:36:21.252 INFO:teuthology.orchestra.run.trial099.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 690 kB/s | 135 kB 00:00 2026-02-20T11:36:21.535 INFO:teuthology.orchestra.run.trial099.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 534 kB/s | 150 kB 00:00 2026-02-20T11:36:21.730 INFO:teuthology.orchestra.run.trial099.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 647 kB/s | 126 kB 00:00 2026-02-20T11:36:22.099 INFO:teuthology.orchestra.run.trial099.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 590 kB/s | 218 kB 00:00 2026-02-20T11:36:22.381 INFO:teuthology.orchestra.run.trial099.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 646 kB/s | 182 kB 00:00 2026-02-20T11:36:22.750 INFO:teuthology.orchestra.run.trial099.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 720 kB/s | 266 kB 00:00 2026-02-20T11:36:23.019 INFO:teuthology.orchestra.run.trial099.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 110 kB/s | 30 kB 00:00 2026-02-20T11:36:23.569 INFO:teuthology.orchestra.run.trial099.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 5.4 MB/s | 3.0 MB 00:00 2026-02-20T11:36:23.663 INFO:teuthology.orchestra.run.trial099.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 157 kB/s | 15 kB 00:00 2026-02-20T11:36:23.762 INFO:teuthology.orchestra.run.trial099.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 2.4 MB/s | 246 kB 00:00 2026-02-20T11:36:23.861 INFO:teuthology.orchestra.run.trial099.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 1.9 MB/s | 188 kB 00:00 2026-02-20T11:36:23.952 INFO:teuthology.orchestra.run.trial099.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 460 kB/s | 42 kB 00:00 2026-02-20T11:36:24.176 INFO:teuthology.orchestra.run.trial099.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 24 MB/s | 5.3 MB 00:00 2026-02-20T11:36:24.425 INFO:teuthology.orchestra.run.trial099.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2026-02-20T11:36:24.530 INFO:teuthology.orchestra.run.trial099.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 2.3 MB/s | 244 kB 00:00 2026-02-20T11:36:24.628 INFO:teuthology.orchestra.run.trial099.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 2.5 MB/s | 249 kB 00:00 2026-02-20T11:36:24.731 INFO:teuthology.orchestra.run.trial099.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 463 kB/s | 48 kB 00:00 2026-02-20T11:36:24.835 INFO:teuthology.orchestra.run.trial099.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 1.7 MB/s | 177 kB 00:00 2026-02-20T11:36:24.943 INFO:teuthology.orchestra.run.trial099.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2026-02-20T11:36:25.047 INFO:teuthology.orchestra.run.trial099.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 335 kB/s | 35 kB 00:00 2026-02-20T11:36:25.123 INFO:teuthology.orchestra.run.trial099.stdout:(44/96): ceph-mgr-dashboard-20.3.0-5357.g0c3148 2.6 MB/s | 51 MB 00:19 2026-02-20T11:36:25.237 INFO:teuthology.orchestra.run.trial099.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 3.8 MB/s | 442 kB 00:00 2026-02-20T11:36:25.319 INFO:teuthology.orchestra.run.trial099.stdout:(46/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 23 MB/s | 6.1 MB 00:00 2026-02-20T11:36:25.338 INFO:teuthology.orchestra.run.trial099.stdout:(47/96): python3-packaging-20.9-5.el9.noarch.rp 770 kB/s | 77 kB 00:00 2026-02-20T11:36:25.422 INFO:teuthology.orchestra.run.trial099.stdout:(48/96): python3-protobuf-3.14.0-17.el9.noarch. 2.6 MB/s | 267 kB 00:00 2026-02-20T11:36:25.437 INFO:teuthology.orchestra.run.trial099.stdout:(49/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2026-02-20T11:36:25.526 INFO:teuthology.orchestra.run.trial099.stdout:(50/96): python3-pyasn1-modules-0.4.8-6.el9.noa 2.6 MB/s | 279 kB 00:00 2026-02-20T11:36:25.535 INFO:teuthology.orchestra.run.trial099.stdout:(51/96): python3-requests-oauthlib-1.3.0-12.el9 548 kB/s | 54 kB 00:00 2026-02-20T11:36:25.640 INFO:teuthology.orchestra.run.trial099.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 398 kB/s | 42 kB 00:00 2026-02-20T11:36:25.787 INFO:teuthology.orchestra.run.trial099.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 1.3 MB/s | 189 kB 00:00 2026-02-20T11:36:25.950 INFO:teuthology.orchestra.run.trial099.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 558 kB/s | 90 kB 00:00 2026-02-20T11:36:26.214 INFO:teuthology.orchestra.run.trial099.stdout:(55/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 28 MB/s | 19 MB 00:00 2026-02-20T11:36:26.490 INFO:teuthology.orchestra.run.trial099.stdout:(56/96): lua-devel-5.4.4-4.el9.x86_64.rpm 41 kB/s | 22 kB 00:00 2026-02-20T11:36:26.758 INFO:teuthology.orchestra.run.trial099.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.0 MB/s | 551 kB 00:00 2026-02-20T11:36:26.782 INFO:teuthology.orchestra.run.trial099.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 801 kB/s | 19 kB 00:00 2026-02-20T11:36:26.809 INFO:teuthology.orchestra.run.trial099.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 5.4 MB/s | 151 kB 00:00 2026-02-20T11:36:26.850 INFO:teuthology.orchestra.run.trial099.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 13 MB/s | 548 kB 00:00 2026-02-20T11:36:26.875 INFO:teuthology.orchestra.run.trial099.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 1.2 MB/s | 29 kB 00:00 2026-02-20T11:36:26.900 INFO:teuthology.orchestra.run.trial099.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 2.3 MB/s | 60 kB 00:00 2026-02-20T11:36:26.925 INFO:teuthology.orchestra.run.trial099.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2026-02-20T11:36:26.950 INFO:teuthology.orchestra.run.trial099.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 1.3 MB/s | 32 kB 00:00 2026-02-20T11:36:26.974 INFO:teuthology.orchestra.run.trial099.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 574 kB/s | 14 kB 00:00 2026-02-20T11:36:27.002 INFO:teuthology.orchestra.run.trial099.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 6.1 MB/s | 173 kB 00:00 2026-02-20T11:36:27.035 INFO:teuthology.orchestra.run.trial099.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 11 MB/s | 358 kB 00:00 2026-02-20T11:36:27.065 INFO:teuthology.orchestra.run.trial099.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 8.4 MB/s | 254 kB 00:00 2026-02-20T11:36:27.117 INFO:teuthology.orchestra.run.trial099.stdout:(69/96): protobuf-compiler-3.14.0-17.el9.x86_64 954 kB/s | 862 kB 00:00 2026-02-20T11:36:27.139 INFO:teuthology.orchestra.run.trial099.stdout:(70/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 28 MB/s | 2.0 MB 00:00 2026-02-20T11:36:27.166 INFO:teuthology.orchestra.run.trial099.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 5.1 MB/s | 139 kB 00:00 2026-02-20T11:36:27.191 INFO:teuthology.orchestra.run.trial099.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 2.2 MB/s | 56 kB 00:00 2026-02-20T11:36:27.216 INFO:teuthology.orchestra.run.trial099.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 435 kB/s | 11 kB 00:00 2026-02-20T11:36:27.241 INFO:teuthology.orchestra.run.trial099.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 717 kB/s | 18 kB 00:00 2026-02-20T11:36:27.266 INFO:teuthology.orchestra.run.trial099.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 941 kB/s | 23 kB 00:00 2026-02-20T11:36:27.284 INFO:teuthology.orchestra.run.trial099.stdout:(76/96): python3-grpcio-tools-1.46.7-10.el9.x86 865 kB/s | 144 kB 00:00 2026-02-20T11:36:27.290 INFO:teuthology.orchestra.run.trial099.stdout:(77/96): python3-jaraco-context-6.0.1-3.el9.noa 803 kB/s | 20 kB 00:00 2026-02-20T11:36:27.315 INFO:teuthology.orchestra.run.trial099.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 1.0 MB/s | 26 kB 00:00 2026-02-20T11:36:27.316 INFO:teuthology.orchestra.run.trial099.stdout:(79/96): python3-jaraco-functools-3.5.0-2.el9.n 601 kB/s | 19 kB 00:00 2026-02-20T11:36:27.348 INFO:teuthology.orchestra.run.trial099.stdout:(80/96): python3-kubernetes-26.1.0-3.el9.noarch 31 MB/s | 1.0 MB 00:00 2026-02-20T11:36:27.357 INFO:teuthology.orchestra.run.trial099.stdout:(81/96): python3-more-itertools-8.12.0-2.el9.no 1.9 MB/s | 79 kB 00:00 2026-02-20T11:36:27.373 INFO:teuthology.orchestra.run.trial099.stdout:(82/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 3.5 MB/s | 86 kB 00:00 2026-02-20T11:36:27.396 INFO:teuthology.orchestra.run.trial099.stdout:(83/96): python3-natsort-7.1.1-5.el9.noarch.rpm 1.5 MB/s | 58 kB 00:00 2026-02-20T11:36:27.396 INFO:teuthology.orchestra.run.trial099.stdout:(84/96): python3-portend-3.1.0-2.el9.noarch.rpm 701 kB/s | 16 kB 00:00 2026-02-20T11:36:27.420 INFO:teuthology.orchestra.run.trial099.stdout:(85/96): python3-repoze-lru-0.7-16.el9.noarch.r 1.3 MB/s | 31 kB 00:00 2026-02-20T11:36:27.438 INFO:teuthology.orchestra.run.trial099.stdout:(86/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2026-02-20T11:36:27.445 INFO:teuthology.orchestra.run.trial099.stdout:(87/96): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2026-02-20T11:36:27.470 INFO:teuthology.orchestra.run.trial099.stdout:(88/96): python3-saml-1.16.0-1.el9.noarch.rpm 5.0 MB/s | 125 kB 00:00 2026-02-20T11:36:27.477 INFO:teuthology.orchestra.run.trial099.stdout:(89/96): python3-rsa-4.9-2.el9.noarch.rpm 1.5 MB/s | 59 kB 00:00 2026-02-20T11:36:27.493 INFO:teuthology.orchestra.run.trial099.stdout:(90/96): python3-tempora-5.0.0-2.el9.noarch.rpm 1.5 MB/s | 36 kB 00:00 2026-02-20T11:36:27.518 INFO:teuthology.orchestra.run.trial099.stdout:(91/96): python3-websocket-client-1.2.3-2.el9.n 3.6 MB/s | 90 kB 00:00 2026-02-20T11:36:27.519 INFO:teuthology.orchestra.run.trial099.stdout:(92/96): python3-typing-extensions-4.15.0-1.el9 2.0 MB/s | 86 kB 00:00 2026-02-20T11:36:27.542 INFO:teuthology.orchestra.run.trial099.stdout:(93/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 2.0 MB/s | 48 kB 00:00 2026-02-20T11:36:27.552 INFO:teuthology.orchestra.run.trial099.stdout:(94/96): python3-xmltodict-0.12.0-15.el9.noarch 684 kB/s | 22 kB 00:00 2026-02-20T11:36:27.565 INFO:teuthology.orchestra.run.trial099.stdout:(95/96): python3-zc-lockfile-2.0-10.el9.noarch. 852 kB/s | 20 kB 00:00 2026-02-20T11:36:44.945 INFO:teuthology.orchestra.run.trial099.stdout:(96/96): ceph-osd-crimson-20.3.0-5357.g0c3148ca 2.2 MB/s | 88 MB 00:40 2026-02-20T11:36:44.948 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:36:44.948 INFO:teuthology.orchestra.run.trial099.stdout:Total 5.3 MB/s | 226 MB 00:42 2026-02-20T11:36:45.336 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:36:45.376 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:36:45.376 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:36:46.176 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:36:46.176 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:36:47.016 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:36:47.018 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-20T11:36:47.022 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-20T11:36:47.043 INFO:teuthology.orchestra.run.trial099.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-20T11:36:47.076 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-20T11:36:47.082 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-20T11:36:47.085 INFO:teuthology.orchestra.run.trial099.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-20T11:36:47.089 INFO:teuthology.orchestra.run.trial099.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-20T11:36:47.112 INFO:teuthology.orchestra.run.trial099.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-20T11:36:47.125 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-20T11:36:47.128 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-20T11:36:47.134 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-20T11:36:47.136 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-20T11:36:47.147 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-20T11:36:47.153 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-20T11:36:47.177 INFO:teuthology.orchestra.run.trial099.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-20T11:36:47.180 INFO:teuthology.orchestra.run.trial099.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-20T11:36:47.230 INFO:teuthology.orchestra.run.trial099.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-20T11:36:47.242 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-20T11:36:47.245 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-20T11:36:47.249 INFO:teuthology.orchestra.run.trial099.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-20T11:36:47.255 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-20T11:36:47.259 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-20T11:36:47.262 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-20T11:36:47.277 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-20T11:36:47.282 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-20T11:36:47.287 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-20T11:36:47.298 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-20T11:36:47.307 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-20T11:36:47.313 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-20T11:36:47.333 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-20T11:36:47.344 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-20T11:36:47.359 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-20T11:36:47.364 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-20T11:36:47.372 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-20T11:36:47.413 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-20T11:36:47.740 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-20T11:36:47.750 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-20T11:36:47.753 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-20T11:36:47.759 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-20T11:36:47.761 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-20T11:36:47.789 INFO:teuthology.orchestra.run.trial099.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-20T11:36:47.833 INFO:teuthology.orchestra.run.trial099.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-20T11:36:47.841 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-20T11:36:47.847 INFO:teuthology.orchestra.run.trial099.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-20T11:36:47.850 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-20T11:36:47.856 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-20T11:36:47.859 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-20T11:36:47.866 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-20T11:36:47.870 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-20T11:36:47.898 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-20T11:36:47.907 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-20T11:36:47.913 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-20T11:36:47.920 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-20T11:36:47.955 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-20T11:36:48.180 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-20T11:36:48.203 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-20T11:36:48.207 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-20T11:36:48.266 INFO:teuthology.orchestra.run.trial099.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-20T11:36:48.268 INFO:teuthology.orchestra.run.trial099.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-20T11:36:48.289 INFO:teuthology.orchestra.run.trial099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-20T11:36:48.616 INFO:teuthology.orchestra.run.trial099.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-20T11:36:48.690 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-20T11:36:49.364 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-20T11:36:49.380 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-20T11:36:49.390 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-20T11:36:49.397 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-20T11:36:49.410 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-20T11:36:49.426 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-20T11:36:49.502 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-20T11:36:49.510 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-20T11:36:49.534 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-20T11:36:49.564 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-20T11:36:49.615 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-20T11:36:49.621 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-20T11:36:49.624 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-20T11:36:49.625 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-20T11:36:49.642 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:36:49.642 INFO:teuthology.orchestra.run.trial099.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-20T11:36:49.642 INFO:teuthology.orchestra.run.trial099.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-20T11:36:49.642 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:36:49.650 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:36:49.675 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:36:49.675 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-20T11:36:49.675 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:36:49.709 INFO:teuthology.orchestra.run.trial099.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-20T11:36:50.295 INFO:teuthology.orchestra.run.trial099.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-20T11:36:55.705 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 80/96 2026-02-20T11:36:55.769 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 81/96 2026-02-20T11:36:55.772 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 82/96 2026-02-20T11:36:55.785 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 82/96 2026-02-20T11:36:55.785 INFO:teuthology.orchestra.run.trial099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:36:55.786 INFO:teuthology.orchestra.run.trial099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-20T11:36:55.786 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:36:55.795 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 83/96 2026-02-20T11:36:55.819 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 83/96 2026-02-20T11:36:55.819 INFO:teuthology.orchestra.run.trial099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:36:55.819 INFO:teuthology.orchestra.run.trial099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-20T11:36:55.819 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T11:36:55.819 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T11:36:55.819 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:36:55.878 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 84/96 2026-02-20T11:36:55.880 INFO:teuthology.orchestra.run.trial099.stdout: Installing : cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 84/96 2026-02-20T11:36:55.890 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9 85/96 2026-02-20T11:36:55.914 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el 86/96 2026-02-20T11:36:55.915 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 87/96 2026-02-20T11:36:58.806 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 87/96 2026-02-20T11:36:58.812 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 88/96 2026-02-20T11:36:59.265 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 88/96 2026-02-20T11:36:59.266 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 89/96 2026-02-20T11:36:59.278 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 89/96 2026-02-20T11:36:59.278 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 90/96 2026-02-20T11:36:59.339 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 90/96 2026-02-20T11:36:59.380 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9. 91/96 2026-02-20T11:36:59.380 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 92/96 2026-02-20T11:36:59.401 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 92/96 2026-02-20T11:36:59.401 INFO:teuthology.orchestra.run.trial099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:36:59.401 INFO:teuthology.orchestra.run.trial099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-20T11:36:59.401 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T11:36:59.402 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T11:36:59.402 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:36:59.413 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 93/96 2026-02-20T11:36:59.628 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 93/96 2026-02-20T11:36:59.630 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 94/96 2026-02-20T11:36:59.651 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 94/96 2026-02-20T11:36:59.651 INFO:teuthology.orchestra.run.trial099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:36:59.651 INFO:teuthology.orchestra.run.trial099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-20T11:36:59.652 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T11:36:59.652 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T11:36:59.652 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:36:59.734 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 95/96 2026-02-20T11:36:59.754 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 95/96 2026-02-20T11:36:59.754 INFO:teuthology.orchestra.run.trial099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:36:59.754 INFO:teuthology.orchestra.run.trial099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-20T11:36:59.754 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T11:36:59.754 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T11:36:59.754 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:36:59.755 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 96/96 2026-02-20T11:36:59.761 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 96/96 2026-02-20T11:36:59.785 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 96/96 2026-02-20T11:37:01.208 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 96/96 2026-02-20T11:37:01.209 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/96 2026-02-20T11:37:01.209 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/96 2026-02-20T11:37:01.209 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/96 2026-02-20T11:37:01.209 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 4/96 2026-02-20T11:37:01.209 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 5/96 2026-02-20T11:37:01.209 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 6/96 2026-02-20T11:37:01.209 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 7/96 2026-02-20T11:37:01.209 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el 8/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 9/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 10/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 11/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 12/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9. 13/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 14/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9 15/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 16/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 17/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-20T11:37:01.210 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-20T11:37:01.211 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-20T11:37:01.212 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-20T11:37:01.213 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-20T11:37:01.214 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-20T11:37:01.215 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-20T11:37:01.216 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-20T11:37:01.216 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-20T11:37:01.216 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-20T11:37:01.216 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-20T11:37:01.216 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-20T11:37:01.216 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-20T11:37:01.216 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-20T11:37:01.216 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-20T11:37:01.216 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-20T11:37:01.216 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-20T11:37:01.216 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-20T11:37:01.272 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-20T11:37:01.273 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:01.273 INFO:teuthology.orchestra.run.trial099.stdout:Installed: 2026-02-20T11:37:01.273 INFO:teuthology.orchestra.run.trial099.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-20T11:37:01.273 INFO:teuthology.orchestra.run.trial099.stdout: ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:01.273 INFO:teuthology.orchestra.run.trial099.stdout: ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:37:01.273 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:01.273 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:01.273 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:37:01.273 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:37:01.273 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:37:01.273 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-20T11:37:01.274 INFO:teuthology.orchestra.run.trial099.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: lua-5.4.4-4.el9.x86_64 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-20T11:37:01.275 INFO:teuthology.orchestra.run.trial099.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-20T11:37:01.276 INFO:teuthology.orchestra.run.trial099.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-20T11:37:01.277 INFO:teuthology.orchestra.run.trial099.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-ply-3.11-14.el9.noarch 2026-02-20T11:37:01.278 INFO:teuthology.orchestra.run.trial099.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-20T11:37:01.279 INFO:teuthology.orchestra.run.trial099.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: unzip-6.0-59.el9.x86_64 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: zip-3.0-35.el9.x86_64 2026-02-20T11:37:01.280 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:01.281 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:01.341 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-base 2026-02-20T11:37:01.657 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:11 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:01.723 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:01.742 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:01.743 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:01.743 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:01.764 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install cephadm 2026-02-20T11:37:02.078 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:12 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:02.144 INFO:teuthology.orchestra.run.trial099.stdout:Package cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:37:02.162 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:02.163 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:02.163 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:02.184 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T11:37:02.501 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:12 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:02.586 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:02.586 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:02.586 INFO:teuthology.orchestra.run.trial099.stdout: Package Arch Version Repo Size 2026-02-20T11:37:02.586 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:02.586 INFO:teuthology.orchestra.run.trial099.stdout:Installing: 2026-02-20T11:37:02.586 INFO:teuthology.orchestra.run.trial099.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 149 k 2026-02-20T11:37:02.586 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:02.586 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:37:02.587 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:02.587 INFO:teuthology.orchestra.run.trial099.stdout:Install 1 Package 2026-02-20T11:37:02.587 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:02.587 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 149 k 2026-02-20T11:37:02.587 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 455 k 2026-02-20T11:37:02.587 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:37:03.068 INFO:teuthology.orchestra.run.trial099.stdout:ceph-immutable-object-cache-20.3.0-5357.g0c3148 310 kB/s | 149 kB 00:00 2026-02-20T11:37:03.069 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:37:03.069 INFO:teuthology.orchestra.run.trial099.stdout:Total 309 kB/s | 149 kB 00:00 2026-02-20T11:37:03.069 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:37:03.074 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:37:03.074 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:37:03.105 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:37:03.106 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:37:03.160 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:37:03.162 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:37:03.185 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:37:03.185 INFO:teuthology.orchestra.run.trial099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:37:03.185 INFO:teuthology.orchestra.run.trial099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-20T11:37:03.186 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:03.468 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:37:03.469 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:03.469 INFO:teuthology.orchestra.run.trial099.stdout:Installed: 2026-02-20T11:37:03.469 INFO:teuthology.orchestra.run.trial099.stdout: ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:03.469 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:03.469 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:03.497 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-mgr 2026-02-20T11:37:03.806 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:13 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:03.871 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:03.890 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:03.890 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:03.890 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:03.912 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T11:37:04.217 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:14 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:04.281 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:37:04.300 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:04.300 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:04.301 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:04.321 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T11:37:04.633 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:14 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:04.698 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:37:04.716 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:04.717 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:04.717 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:04.737 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-mgr-rook 2026-02-20T11:37:05.043 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:15 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:05.107 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:37:05.126 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:05.126 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:05.127 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:05.147 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T11:37:05.460 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:15 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:05.523 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:37:05.542 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:05.542 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:05.543 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:05.564 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-osd 2026-02-20T11:37:05.876 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:15 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:05.942 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:05.961 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:05.961 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:05.961 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:05.983 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-osd-classic 2026-02-20T11:37:06.290 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:16 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:06.354 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:06.371 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:06.372 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:06.372 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:06.392 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-fuse 2026-02-20T11:37:06.702 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:16 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:06.787 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:06.787 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:06.787 INFO:teuthology.orchestra.run.trial099.stdout: Package Arch Version Repo Size 2026-02-20T11:37:06.787 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:06.787 INFO:teuthology.orchestra.run.trial099.stdout:Installing: 2026-02-20T11:37:06.787 INFO:teuthology.orchestra.run.trial099.stdout: ceph-fuse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 914 k 2026-02-20T11:37:06.787 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:06.788 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:37:06.788 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:06.788 INFO:teuthology.orchestra.run.trial099.stdout:Install 1 Package 2026-02-20T11:37:06.788 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:06.788 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 914 k 2026-02-20T11:37:06.788 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 2.9 M 2026-02-20T11:37:06.788 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:37:07.467 INFO:teuthology.orchestra.run.trial099.stdout:ceph-fuse-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 1.3 MB/s | 914 kB 00:00 2026-02-20T11:37:07.467 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:37:07.467 INFO:teuthology.orchestra.run.trial099.stdout:Total 1.3 MB/s | 914 kB 00:00 2026-02-20T11:37:07.467 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:37:07.472 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:37:07.472 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:37:07.505 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:37:07.506 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:37:07.595 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:37:07.612 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:07.823 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:07.880 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:07.880 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:07.880 INFO:teuthology.orchestra.run.trial099.stdout:Installed: 2026-02-20T11:37:07.880 INFO:teuthology.orchestra.run.trial099.stdout: ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:07.881 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:07.881 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:07.908 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-volume 2026-02-20T11:37:08.220 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:18 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:08.285 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:37:08.304 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:08.304 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:08.304 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:08.326 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install librados-devel 2026-02-20T11:37:08.638 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:18 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:08.722 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:08.723 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:08.723 INFO:teuthology.orchestra.run.trial099.stdout: Package Arch Version Repo Size 2026-02-20T11:37:08.723 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:08.723 INFO:teuthology.orchestra.run.trial099.stdout:Installing: 2026-02-20T11:37:08.723 INFO:teuthology.orchestra.run.trial099.stdout: librados-devel x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 124 k 2026-02-20T11:37:08.723 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:08.723 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:37:08.723 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:08.723 INFO:teuthology.orchestra.run.trial099.stdout:Install 1 Package 2026-02-20T11:37:08.724 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:08.724 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 124 k 2026-02-20T11:37:08.724 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 434 k 2026-02-20T11:37:08.724 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:37:09.213 INFO:teuthology.orchestra.run.trial099.stdout:librados-devel-20.3.0-5357.g0c3148ca.el9.x86_64 253 kB/s | 124 kB 00:00 2026-02-20T11:37:09.213 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:37:09.213 INFO:teuthology.orchestra.run.trial099.stdout:Total 252 kB/s | 124 kB 00:00 2026-02-20T11:37:09.214 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:37:09.216 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:37:09.216 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:37:09.251 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:37:09.252 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:37:09.307 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:37:09.323 INFO:teuthology.orchestra.run.trial099.stdout: Installing : librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:09.406 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:09.465 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:09.466 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:09.466 INFO:teuthology.orchestra.run.trial099.stdout:Installed: 2026-02-20T11:37:09.466 INFO:teuthology.orchestra.run.trial099.stdout: librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:09.466 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:09.466 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:09.495 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install libcephfs2 2026-02-20T11:37:09.801 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:19 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:09.867 INFO:teuthology.orchestra.run.trial099.stdout:Package libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:09.885 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:09.885 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:09.885 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:09.906 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install libcephfs-devel 2026-02-20T11:37:10.216 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:20 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:10.298 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:10.298 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:10.298 INFO:teuthology.orchestra.run.trial099.stdout: Package Arch Version Repo Size 2026-02-20T11:37:10.298 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:10.298 INFO:teuthology.orchestra.run.trial099.stdout:Installing: 2026-02-20T11:37:10.299 INFO:teuthology.orchestra.run.trial099.stdout: libcephfs-devel x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 37 k 2026-02-20T11:37:10.299 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:10.299 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:37:10.299 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:10.299 INFO:teuthology.orchestra.run.trial099.stdout:Install 1 Package 2026-02-20T11:37:10.299 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:10.299 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 37 k 2026-02-20T11:37:10.299 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 171 k 2026-02-20T11:37:10.299 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:37:10.643 INFO:teuthology.orchestra.run.trial099.stdout:libcephfs-devel-20.3.0-5357.g0c3148ca.el9.x86_6 109 kB/s | 37 kB 00:00 2026-02-20T11:37:10.643 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:37:10.643 INFO:teuthology.orchestra.run.trial099.stdout:Total 109 kB/s | 37 kB 00:00 2026-02-20T11:37:10.643 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:37:10.645 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:37:10.645 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:37:10.648 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:37:10.648 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:37:10.669 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:37:10.685 INFO:teuthology.orchestra.run.trial099.stdout: Installing : libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:10.741 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:10.798 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:10.798 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:10.798 INFO:teuthology.orchestra.run.trial099.stdout:Installed: 2026-02-20T11:37:10.798 INFO:teuthology.orchestra.run.trial099.stdout: libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:10.798 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:10.799 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:10.826 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install librados2 2026-02-20T11:37:11.137 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:21 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:11.202 INFO:teuthology.orchestra.run.trial099.stdout:Package librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:11.220 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:11.221 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:11.221 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:11.243 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install librbd1 2026-02-20T11:37:11.555 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:21 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:11.619 INFO:teuthology.orchestra.run.trial099.stdout:Package librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:11.638 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:11.638 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:11.638 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:11.660 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install python3-rados 2026-02-20T11:37:11.966 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:21 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:12.032 INFO:teuthology.orchestra.run.trial099.stdout:Package python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:12.050 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:12.051 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:12.051 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:12.072 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install python3-rgw 2026-02-20T11:37:12.380 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:22 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:12.445 INFO:teuthology.orchestra.run.trial099.stdout:Package python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:12.464 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:12.464 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:12.464 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:12.486 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install python3-cephfs 2026-02-20T11:37:12.798 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:22 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:12.862 INFO:teuthology.orchestra.run.trial099.stdout:Package python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:12.880 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:12.881 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:12.881 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:12.903 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install python3-rbd 2026-02-20T11:37:13.215 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:23 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:13.280 INFO:teuthology.orchestra.run.trial099.stdout:Package python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:13.299 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:13.299 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:13.300 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:13.321 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install rbd-fuse 2026-02-20T11:37:13.628 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:23 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:13.714 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:13.714 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:13.714 INFO:teuthology.orchestra.run.trial099.stdout: Package Architecture Version Repository Size 2026-02-20T11:37:13.714 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:13.714 INFO:teuthology.orchestra.run.trial099.stdout:Installing: 2026-02-20T11:37:13.714 INFO:teuthology.orchestra.run.trial099.stdout: rbd-fuse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 86 k 2026-02-20T11:37:13.714 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:13.715 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:37:13.715 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:13.715 INFO:teuthology.orchestra.run.trial099.stdout:Install 1 Package 2026-02-20T11:37:13.715 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:13.715 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 86 k 2026-02-20T11:37:13.715 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 236 k 2026-02-20T11:37:13.715 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:37:14.325 INFO:teuthology.orchestra.run.trial099.stdout:rbd-fuse-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 141 kB/s | 86 kB 00:00 2026-02-20T11:37:14.326 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:37:14.326 INFO:teuthology.orchestra.run.trial099.stdout:Total 141 kB/s | 86 kB 00:00 2026-02-20T11:37:14.326 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:37:14.330 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:37:14.330 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:37:14.361 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:37:14.361 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:37:14.412 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:37:14.429 INFO:teuthology.orchestra.run.trial099.stdout: Installing : rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:14.510 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:14.567 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:14.568 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:14.568 INFO:teuthology.orchestra.run.trial099.stdout:Installed: 2026-02-20T11:37:14.568 INFO:teuthology.orchestra.run.trial099.stdout: rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:14.568 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:14.568 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:14.596 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install rbd-mirror 2026-02-20T11:37:14.913 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:24 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:14.998 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:14.998 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout: Package Arch Version Repo Size 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout:Installing: 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout: rbd-mirror x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.6 M 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout:Install 1 Package 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 2.6 M 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 11 M 2026-02-20T11:37:14.999 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:37:16.211 INFO:teuthology.orchestra.run.trial099.stdout:rbd-mirror-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 2.1 MB/s | 2.6 MB 00:01 2026-02-20T11:37:16.211 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:37:16.211 INFO:teuthology.orchestra.run.trial099.stdout:Total 2.1 MB/s | 2.6 MB 00:01 2026-02-20T11:37:16.211 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:37:16.216 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:37:16.216 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:37:16.256 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:37:16.256 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:37:16.420 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:37:16.421 INFO:teuthology.orchestra.run.trial099.stdout: Installing : rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:16.445 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:16.445 INFO:teuthology.orchestra.run.trial099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:37:16.445 INFO:teuthology.orchestra.run.trial099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-20T11:37:16.445 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T11:37:16.445 INFO:teuthology.orchestra.run.trial099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T11:37:16.445 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:16.765 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:16.765 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:16.765 INFO:teuthology.orchestra.run.trial099.stdout:Installed: 2026-02-20T11:37:16.765 INFO:teuthology.orchestra.run.trial099.stdout: rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:16.765 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:16.765 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:16.794 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install rbd-nbd 2026-02-20T11:37:17.110 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:27 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:17.195 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:17.195 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:17.195 INFO:teuthology.orchestra.run.trial099.stdout: Package Architecture Version Repository Size 2026-02-20T11:37:17.195 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:17.195 INFO:teuthology.orchestra.run.trial099.stdout:Installing: 2026-02-20T11:37:17.195 INFO:teuthology.orchestra.run.trial099.stdout: rbd-nbd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 167 k 2026-02-20T11:37:17.195 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:17.196 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:37:17.196 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:17.196 INFO:teuthology.orchestra.run.trial099.stdout:Install 1 Package 2026-02-20T11:37:17.196 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:17.196 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 167 k 2026-02-20T11:37:17.196 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 496 k 2026-02-20T11:37:17.196 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:37:17.612 INFO:teuthology.orchestra.run.trial099.stdout:rbd-nbd-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 400 kB/s | 167 kB 00:00 2026-02-20T11:37:17.613 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:37:17.613 INFO:teuthology.orchestra.run.trial099.stdout:Total 399 kB/s | 167 kB 00:00 2026-02-20T11:37:17.613 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:37:17.618 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:37:17.618 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:37:17.652 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:37:17.652 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:37:17.711 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:37:17.728 INFO:teuthology.orchestra.run.trial099.stdout: Installing : rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:17.808 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:17.870 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:37:17.870 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:17.871 INFO:teuthology.orchestra.run.trial099.stdout:Installed: 2026-02-20T11:37:17.871 INFO:teuthology.orchestra.run.trial099.stdout: rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:37:17.871 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:17.871 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:17.896 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-osd-crimson 2026-02-20T11:37:18.219 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:28 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:18.283 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:37:18.301 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:18.302 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:37:18.302 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:37:18.324 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-20T11:37:18.637 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:02:28 ago on Fri 20 Feb 2026 11:34:50 AM UTC. 2026-02-20T11:37:18.720 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:37:18.721 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:18.721 INFO:teuthology.orchestra.run.trial099.stdout: Package Arch Version Repo Size 2026-02-20T11:37:18.721 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:18.721 INFO:teuthology.orchestra.run.trial099.stdout:Installing: 2026-02-20T11:37:18.721 INFO:teuthology.orchestra.run.trial099.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 433 M 2026-02-20T11:37:18.721 INFO:teuthology.orchestra.run.trial099.stdout:Installing weak dependencies: 2026-02-20T11:37:18.721 INFO:teuthology.orchestra.run.trial099.stdout: ceph-debugsource x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 12 M 2026-02-20T11:37:18.721 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:18.721 INFO:teuthology.orchestra.run.trial099.stdout:Transaction Summary 2026-02-20T11:37:18.722 INFO:teuthology.orchestra.run.trial099.stdout:================================================================================ 2026-02-20T11:37:18.722 INFO:teuthology.orchestra.run.trial099.stdout:Install 2 Packages 2026-02-20T11:37:18.722 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:37:18.722 INFO:teuthology.orchestra.run.trial099.stdout:Total download size: 445 M 2026-02-20T11:37:18.722 INFO:teuthology.orchestra.run.trial099.stdout:Installed size: 2.7 G 2026-02-20T11:37:18.722 INFO:teuthology.orchestra.run.trial099.stdout:Downloading Packages: 2026-02-20T11:37:22.415 INFO:teuthology.orchestra.run.trial099.stdout:(1/2): ceph-debugsource-20.3.0-5357.g0c3148ca.e 3.3 MB/s | 12 MB 00:03 2026-02-20T11:41:12.563 INFO:teuthology.orchestra.run.trial099.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5357.g 1.9 MB/s | 433 MB 03:53 2026-02-20T11:41:12.569 INFO:teuthology.orchestra.run.trial099.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:41:12.569 INFO:teuthology.orchestra.run.trial099.stdout:Total 1.9 MB/s | 445 MB 03:53 2026-02-20T11:41:12.569 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction check 2026-02-20T11:41:12.570 INFO:teuthology.orchestra.run.trial099.stdout:Transaction check succeeded. 2026-02-20T11:41:12.570 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction test 2026-02-20T11:41:13.950 INFO:teuthology.orchestra.run.trial099.stdout:Transaction test succeeded. 2026-02-20T11:41:13.950 INFO:teuthology.orchestra.run.trial099.stdout:Running transaction 2026-02-20T11:41:17.550 INFO:teuthology.orchestra.run.trial099.stdout: Preparing : 1/1 2026-02-20T11:41:39.763 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/2 2026-02-20T11:41:39.788 INFO:teuthology.orchestra.run.trial099.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:41:41.347 INFO:teuthology.orchestra.run.trial099.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:41:41.348 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/2 2026-02-20T11:41:41.406 INFO:teuthology.orchestra.run.trial099.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:41:41.406 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:41:41.406 INFO:teuthology.orchestra.run.trial099.stdout:Installed: 2026-02-20T11:41:41.406 INFO:teuthology.orchestra.run.trial099.stdout: ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:41:41.406 INFO:teuthology.orchestra.run.trial099.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:41:41.407 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:41:41.407 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:41:41.489 DEBUG:teuthology.parallel:result is None 2026-02-20T11:41:41.491 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:41:41.740 DEBUG:teuthology.orchestra.run.trial099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T11:41:41.761 INFO:teuthology.orchestra.run.trial099.stdout:20.3.0-5357.g0c3148ca.el9 2026-02-20T11:41:41.762 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5357.g0c3148ca.el9 2026-02-20T11:41:41.762 INFO:teuthology.task.install:The correct ceph version 20.3.0-5357.g0c3148ca is installed. 2026-02-20T11:41:41.766 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-20T11:41:41.766 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:41:41.766 DEBUG:teuthology.orchestra.run.trial099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-20T11:41:41.829 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-20T11:41:41.829 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:41:41.829 DEBUG:teuthology.orchestra.run.trial099:> sudo dd of=/usr/bin/daemon-helper 2026-02-20T11:41:41.893 DEBUG:teuthology.orchestra.run.trial099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-20T11:41:41.959 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-20T11:41:41.959 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:41:41.959 DEBUG:teuthology.orchestra.run.trial099:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-20T11:41:42.024 DEBUG:teuthology.orchestra.run.trial099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-20T11:41:42.090 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-20T11:41:42.090 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:41:42.090 DEBUG:teuthology.orchestra.run.trial099:> sudo dd of=/usr/bin/stdin-killer 2026-02-20T11:41:42.154 DEBUG:teuthology.orchestra.run.trial099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-20T11:41:42.219 INFO:teuthology.run_tasks:Running task pexec... 2026-02-20T11:41:42.237 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-20T11:41:42.237 DEBUG:teuthology.orchestra.run.trial099:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-20T11:41:42.262 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial099.front.sepia.ceph.com 2026-02-20T11:41:42.262 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-20T11:41:42.312 DEBUG:teuthology.parallel:result is None 2026-02-20T11:41:42.312 INFO:teuthology.run_tasks:Running task install... 2026-02-20T11:41:42.325 DEBUG:teuthology.task.install:project ceph 2026-02-20T11:41:42.326 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}} 2026-02-20T11:41:42.326 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'} 2026-02-20T11:41:42.326 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T11:41:42.335 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T11:41:42.335 INFO:teuthology.task.install:extra packages: [] 2026-02-20T11:41:42.336 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'tag': None, 'wait_for_package': False} 2026-02-20T11:41:42.336 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:41:42.494 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/ 2026-02-20T11:41:42.495 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5357.g0c3148ca 2026-02-20T11:41:42.879 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T11:41:42.879 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:41:42.880 DEBUG:teuthology.orchestra.run.trial099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T11:41:42.904 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-20T11:41:42.904 DEBUG:teuthology.orchestra.run.trial099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T11:41:42.971 DEBUG:teuthology.orchestra.run.trial099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T11:41:43.035 DEBUG:teuthology.orchestra.run.trial099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T11:41:43.092 INFO:teuthology.orchestra.run.trial099.stdout:check_obsoletes = 1 2026-02-20T11:41:43.101 DEBUG:teuthology.orchestra.run.trial099:> sudo yum clean all 2026-02-20T11:41:43.260 INFO:teuthology.orchestra.run.trial099.stdout:56 files removed 2026-02-20T11:41:43.274 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-radosgw 2026-02-20T11:41:44.137 INFO:teuthology.orchestra.run.trial099.stdout:ceph packages for x86_64 131 kB/s | 94 kB 00:00 2026-02-20T11:41:44.868 INFO:teuthology.orchestra.run.trial099.stdout:ceph noarch packages 35 kB/s | 24 kB 00:00 2026-02-20T11:41:45.391 INFO:teuthology.orchestra.run.trial099.stdout:ceph source packages 4.1 kB/s | 2.1 kB 00:00 2026-02-20T11:41:46.804 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - BaseOS 6.4 MB/s | 8.9 MB 00:01 2026-02-20T11:41:49.880 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-20T11:41:53.255 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 7.8 MB 00:01 2026-02-20T11:41:54.206 INFO:teuthology.orchestra.run.trial099.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 20 kB 00:00 2026-02-20T11:41:55.001 INFO:teuthology.orchestra.run.trial099.stdout:Extra Packages for Enterprise Linux 28 MB/s | 20 MB 00:00 2026-02-20T11:41:58.405 INFO:teuthology.orchestra.run.trial099.stdout:lab-extras 273 kB/s | 50 kB 00:00 2026-02-20T11:41:59.599 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:41:59.618 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:41:59.618 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:41:59.618 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:41:59.642 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-test 2026-02-20T11:41:59.957 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:00.023 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:00.041 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:00.042 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:00.042 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:00.064 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph 2026-02-20T11:42:00.380 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:00.446 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:00.465 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:00.466 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:00.466 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:00.488 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-base 2026-02-20T11:42:00.803 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:00.868 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:00.887 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:00.887 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:00.887 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:00.909 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install cephadm 2026-02-20T11:42:01.223 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:01.287 INFO:teuthology.orchestra.run.trial099.stdout:Package cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:42:01.306 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:01.307 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:01.307 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:01.328 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T11:42:01.644 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:01.710 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:01.728 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:01.729 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:01.729 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:01.750 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-mgr 2026-02-20T11:42:02.061 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:02.127 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:02.145 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:02.146 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:02.146 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:02.168 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T11:42:02.484 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:02.549 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:42:02.567 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:02.568 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:02.568 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:02.589 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T11:42:02.904 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:02.969 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:42:02.988 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:02.989 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:02.989 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:03.011 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-mgr-rook 2026-02-20T11:42:03.323 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:05 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:03.388 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:42:03.406 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:03.407 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:03.407 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:03.430 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T11:42:03.738 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:05 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:03.802 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:42:03.821 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:03.821 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:03.821 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:03.842 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-osd 2026-02-20T11:42:04.155 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:04.220 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:04.238 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:04.239 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:04.239 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:04.261 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-osd-classic 2026-02-20T11:42:04.586 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:04.651 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:04.670 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:04.671 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:04.671 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:04.694 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-fuse 2026-02-20T11:42:05.002 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:05.067 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:05.085 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:05.086 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:05.086 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:05.107 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install ceph-volume 2026-02-20T11:42:05.416 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:05.481 INFO:teuthology.orchestra.run.trial099.stdout:Package ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:42:05.499 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:05.500 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:05.500 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:05.522 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install librados-devel 2026-02-20T11:42:05.834 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:05.899 INFO:teuthology.orchestra.run.trial099.stdout:Package librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:05.917 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:05.918 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:05.918 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:05.940 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install libcephfs2 2026-02-20T11:42:06.248 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:08 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:06.314 INFO:teuthology.orchestra.run.trial099.stdout:Package libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:06.332 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:06.333 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:06.333 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:06.355 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install libcephfs-devel 2026-02-20T11:42:06.671 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:08 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:06.736 INFO:teuthology.orchestra.run.trial099.stdout:Package libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:06.755 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:06.755 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:06.755 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:06.777 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install librados2 2026-02-20T11:42:07.087 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:07.153 INFO:teuthology.orchestra.run.trial099.stdout:Package librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:07.172 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:07.173 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:07.173 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:07.195 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install librbd1 2026-02-20T11:42:07.505 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:07.569 INFO:teuthology.orchestra.run.trial099.stdout:Package librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:07.588 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:07.589 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:07.589 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:07.609 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install python3-rados 2026-02-20T11:42:07.916 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:07.981 INFO:teuthology.orchestra.run.trial099.stdout:Package python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:08.000 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:08.000 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:08.000 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:08.022 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install python3-rgw 2026-02-20T11:42:08.336 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:10 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:08.401 INFO:teuthology.orchestra.run.trial099.stdout:Package python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:08.420 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:08.420 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:08.420 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:08.443 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install python3-cephfs 2026-02-20T11:42:08.755 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:10 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:08.819 INFO:teuthology.orchestra.run.trial099.stdout:Package python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:08.838 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:08.839 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:08.839 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:08.860 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install python3-rbd 2026-02-20T11:42:09.174 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:09.239 INFO:teuthology.orchestra.run.trial099.stdout:Package python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:09.258 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:09.258 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:09.258 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:09.280 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install rbd-fuse 2026-02-20T11:42:09.590 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:09.657 INFO:teuthology.orchestra.run.trial099.stdout:Package rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:09.675 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:09.676 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:09.676 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:09.699 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install rbd-mirror 2026-02-20T11:42:10.013 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:12 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:10.078 INFO:teuthology.orchestra.run.trial099.stdout:Package rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:10.097 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:10.097 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:10.097 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:10.119 DEBUG:teuthology.orchestra.run.trial099:> sudo yum -y install rbd-nbd 2026-02-20T11:42:10.427 INFO:teuthology.orchestra.run.trial099.stdout:Last metadata expiration check: 0:00:12 ago on Fri 20 Feb 2026 11:41:58 AM UTC. 2026-02-20T11:42:10.492 INFO:teuthology.orchestra.run.trial099.stdout:Package rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:42:10.511 INFO:teuthology.orchestra.run.trial099.stdout:Dependencies resolved. 2026-02-20T11:42:10.511 INFO:teuthology.orchestra.run.trial099.stdout:Nothing to do. 2026-02-20T11:42:10.511 INFO:teuthology.orchestra.run.trial099.stdout:Complete! 2026-02-20T11:42:10.536 DEBUG:teuthology.parallel:result is None 2026-02-20T11:42:10.536 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:42:10.710 DEBUG:teuthology.orchestra.run.trial099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T11:42:10.732 INFO:teuthology.orchestra.run.trial099.stdout:20.3.0-5357.g0c3148ca.el9 2026-02-20T11:42:10.732 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5357.g0c3148ca.el9 2026-02-20T11:42:10.733 INFO:teuthology.task.install:The correct ceph version 20.3.0-5357.g0c3148ca is installed. 2026-02-20T11:42:10.736 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-20T11:42:10.736 INFO:teuthology.run_tasks:Running task ceph... 2026-02-20T11:42:10.841 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-20T11:42:10.842 DEBUG:teuthology.orchestra.run.trial099:> sudo chmod 777 /var/log/ceph 2026-02-20T11:42:10.867 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-20T11:42:10.867 DEBUG:teuthology.orchestra.run.trial099:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-20T11:42:10.930 INFO:tasks.ceph:Creating extra log directories... 2026-02-20T11:42:10.930 DEBUG:teuthology.orchestra.run.trial099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-20T11:42:11.006 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-20T11:42:11.006 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-20T11:42:11.006 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60546', 'branch': 'wip-perezjos-crimson-only-19-02-2026-PR64439', 'description': 'crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rbd_lock_and_fence}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '60546', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-perezjos-crimson-only-19-02-2026-PR64439'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '445d5c9ba18a4040e70e25303230c36c77773acb'}}, 'owner': 'scheduled_jjperez@soko04.front.sepia.ceph.com', 'priority': 101, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 1736, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '445d5c9ba18a4040e70e25303230c36c77773acb', 'targets': {'trial099.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLeO7dFB75VWN+Jk4X2HfPPHd0/rSpRilN07tGug2dA8CXWfWX6dhjWKP5rpaawg1/5dPbSRnMOidpRXtE2FFTw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rbd/test_lock_fence.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'reimage-ssh-down', 'teuthology_sha1': 'e6d9099ce03dc21193fda653159ec82465c3a94a', 'timestamp': '2026-02-20_10:21:31', 'tube': 'trial', 'user': 'jjperez', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-20T11:42:11.007 DEBUG:teuthology.orchestra.run.trial099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-20T11:42:11.051 DEBUG:teuthology.orchestra.run.trial099:> sudo install -d -m0777 -- /var/run/ceph 2026-02-20T11:42:11.116 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:11.116 DEBUG:teuthology.orchestra.run.trial099:> dd if=/scratch_devs of=/dev/stdout 2026-02-20T11:42:11.173 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-20T11:42:11.174 DEBUG:teuthology.orchestra.run.trial099:> stat /dev/vg_nvme/lv_1 2026-02-20T11:42:11.230 INFO:teuthology.orchestra.run.trial099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-20T11:42:11.230 INFO:teuthology.orchestra.run.trial099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:42:11.230 INFO:teuthology.orchestra.run.trial099.stdout:Device: 6h/6d Inode: 1319 Links: 1 2026-02-20T11:42:11.230 INFO:teuthology.orchestra.run.trial099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:42:11.230 INFO:teuthology.orchestra.run.trial099.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:42:11.231 INFO:teuthology.orchestra.run.trial099.stdout:Access: 2026-02-20 11:37:16.585484623 +0000 2026-02-20T11:42:11.231 INFO:teuthology.orchestra.run.trial099.stdout:Modify: 2026-02-20 11:35:04.768996867 +0000 2026-02-20T11:42:11.231 INFO:teuthology.orchestra.run.trial099.stdout:Change: 2026-02-20 11:35:04.768996867 +0000 2026-02-20T11:42:11.231 INFO:teuthology.orchestra.run.trial099.stdout: Birth: 2026-02-20 11:35:04.768996867 +0000 2026-02-20T11:42:11.231 DEBUG:teuthology.orchestra.run.trial099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-20T11:42:11.293 INFO:teuthology.orchestra.run.trial099.stderr:1+0 records in 2026-02-20T11:42:11.293 INFO:teuthology.orchestra.run.trial099.stderr:1+0 records out 2026-02-20T11:42:11.293 INFO:teuthology.orchestra.run.trial099.stderr:512 bytes copied, 0.000164705 s, 3.1 MB/s 2026-02-20T11:42:11.295 DEBUG:teuthology.orchestra.run.trial099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-20T11:42:11.352 DEBUG:teuthology.orchestra.run.trial099:> stat /dev/vg_nvme/lv_2 2026-02-20T11:42:11.408 INFO:teuthology.orchestra.run.trial099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-20T11:42:11.408 INFO:teuthology.orchestra.run.trial099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:42:11.408 INFO:teuthology.orchestra.run.trial099.stdout:Device: 6h/6d Inode: 1311 Links: 1 2026-02-20T11:42:11.408 INFO:teuthology.orchestra.run.trial099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:42:11.408 INFO:teuthology.orchestra.run.trial099.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:42:11.408 INFO:teuthology.orchestra.run.trial099.stdout:Access: 2026-02-20 11:37:16.585484623 +0000 2026-02-20T11:42:11.409 INFO:teuthology.orchestra.run.trial099.stdout:Modify: 2026-02-20 11:35:04.767996906 +0000 2026-02-20T11:42:11.409 INFO:teuthology.orchestra.run.trial099.stdout:Change: 2026-02-20 11:35:04.767996906 +0000 2026-02-20T11:42:11.409 INFO:teuthology.orchestra.run.trial099.stdout: Birth: 2026-02-20 11:35:04.767996906 +0000 2026-02-20T11:42:11.409 DEBUG:teuthology.orchestra.run.trial099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-20T11:42:11.472 INFO:teuthology.orchestra.run.trial099.stderr:1+0 records in 2026-02-20T11:42:11.472 INFO:teuthology.orchestra.run.trial099.stderr:1+0 records out 2026-02-20T11:42:11.472 INFO:teuthology.orchestra.run.trial099.stderr:512 bytes copied, 0.000143687 s, 3.6 MB/s 2026-02-20T11:42:11.473 DEBUG:teuthology.orchestra.run.trial099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-20T11:42:11.531 DEBUG:teuthology.orchestra.run.trial099:> stat /dev/vg_nvme/lv_3 2026-02-20T11:42:11.589 INFO:teuthology.orchestra.run.trial099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-20T11:42:11.589 INFO:teuthology.orchestra.run.trial099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:42:11.589 INFO:teuthology.orchestra.run.trial099.stdout:Device: 6h/6d Inode: 1298 Links: 1 2026-02-20T11:42:11.589 INFO:teuthology.orchestra.run.trial099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:42:11.589 INFO:teuthology.orchestra.run.trial099.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:42:11.590 INFO:teuthology.orchestra.run.trial099.stdout:Access: 2026-02-20 11:37:16.585484623 +0000 2026-02-20T11:42:11.590 INFO:teuthology.orchestra.run.trial099.stdout:Modify: 2026-02-20 11:35:04.767996906 +0000 2026-02-20T11:42:11.590 INFO:teuthology.orchestra.run.trial099.stdout:Change: 2026-02-20 11:35:04.767996906 +0000 2026-02-20T11:42:11.590 INFO:teuthology.orchestra.run.trial099.stdout: Birth: 2026-02-20 11:35:04.767996906 +0000 2026-02-20T11:42:11.590 DEBUG:teuthology.orchestra.run.trial099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-20T11:42:11.652 INFO:teuthology.orchestra.run.trial099.stderr:1+0 records in 2026-02-20T11:42:11.652 INFO:teuthology.orchestra.run.trial099.stderr:1+0 records out 2026-02-20T11:42:11.652 INFO:teuthology.orchestra.run.trial099.stderr:512 bytes copied, 0.000146522 s, 3.5 MB/s 2026-02-20T11:42:11.654 DEBUG:teuthology.orchestra.run.trial099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-20T11:42:11.710 DEBUG:teuthology.orchestra.run.trial099:> stat /dev/vg_nvme/lv_4 2026-02-20T11:42:11.767 INFO:teuthology.orchestra.run.trial099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-20T11:42:11.767 INFO:teuthology.orchestra.run.trial099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:42:11.767 INFO:teuthology.orchestra.run.trial099.stdout:Device: 6h/6d Inode: 1304 Links: 1 2026-02-20T11:42:11.767 INFO:teuthology.orchestra.run.trial099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:42:11.767 INFO:teuthology.orchestra.run.trial099.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:42:11.767 INFO:teuthology.orchestra.run.trial099.stdout:Access: 2026-02-20 11:37:16.586484619 +0000 2026-02-20T11:42:11.767 INFO:teuthology.orchestra.run.trial099.stdout:Modify: 2026-02-20 11:35:04.767996906 +0000 2026-02-20T11:42:11.768 INFO:teuthology.orchestra.run.trial099.stdout:Change: 2026-02-20 11:35:04.767996906 +0000 2026-02-20T11:42:11.768 INFO:teuthology.orchestra.run.trial099.stdout: Birth: 2026-02-20 11:35:04.767996906 +0000 2026-02-20T11:42:11.768 DEBUG:teuthology.orchestra.run.trial099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-20T11:42:11.829 INFO:teuthology.orchestra.run.trial099.stderr:1+0 records in 2026-02-20T11:42:11.829 INFO:teuthology.orchestra.run.trial099.stderr:1+0 records out 2026-02-20T11:42:11.830 INFO:teuthology.orchestra.run.trial099.stderr:512 bytes copied, 0.000175666 s, 2.9 MB/s 2026-02-20T11:42:11.831 DEBUG:teuthology.orchestra.run.trial099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-20T11:42:11.887 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:42:11.888 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T11:42:11.888 INFO:tasks.ceph:Generating config... 2026-02-20T11:42:11.891 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-20T11:42:11.891 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-20T11:42:11.891 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-20T11:42:11.891 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-20T11:42:11.891 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-20T11:42:11.891 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-20T11:42:11.891 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-20T11:42:11.891 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-20T11:42:11.891 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-20T11:42:11.891 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-20T11:42:11.892 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-20T11:42:11.893 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-20T11:42:11.893 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-20T11:42:11.893 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-20T11:42:11.893 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-20T11:42:11.893 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-20T11:42:11.893 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-20T11:42:11.893 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-20T11:42:11.893 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-20T11:42:11.893 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-20T11:42:11.893 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-20T11:42:11.893 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-20T11:42:11.893 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-20T11:42:11.893 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-20T11:42:11.894 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-20T11:42:11.894 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-20T11:42:11.894 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-20T11:42:11.894 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-20T11:42:11.894 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-20T11:42:11.894 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-20T11:42:11.894 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-20T11:42:11.894 INFO:tasks.ceph:Setting up mon.a... 2026-02-20T11:42:11.895 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-20T11:42:11.968 INFO:teuthology.orchestra.run.trial099.stdout:creating /etc/ceph/ceph.keyring 2026-02-20T11:42:11.971 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-20T11:42:12.055 DEBUG:teuthology.orchestra.run.trial099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T11:42:12.121 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.99')] 2026-02-20T11:42:12.122 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.99', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-20T11:42:12.123 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:12.123 DEBUG:teuthology.orchestra.run.trial099:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T11:42:12.177 DEBUG:teuthology.orchestra.run.trial099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.99 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:42:12.257 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:12.255+0000 7fbcbb240f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:12.257 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:12.255+0000 7fbcbb240f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:12.257 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:12.255+0000 7fbcbb240f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:12.258 INFO:teuthology.orchestra.run.trial099.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:42:12.258 INFO:teuthology.orchestra.run.trial099.stdout:monmaptool: generated fsid 90b0dec6-5a72-419e-80f0-d25365c72a06 2026-02-20T11:42:12.259 INFO:teuthology.orchestra.run.trial099.stdout:setting min_mon_release = tentacle 2026-02-20T11:42:12.259 INFO:teuthology.orchestra.run.trial099.stdout:epoch 0 2026-02-20T11:42:12.259 INFO:teuthology.orchestra.run.trial099.stdout:fsid 90b0dec6-5a72-419e-80f0-d25365c72a06 2026-02-20T11:42:12.259 INFO:teuthology.orchestra.run.trial099.stdout:last_changed 2026-02-20T11:42:12.256417+0000 2026-02-20T11:42:12.259 INFO:teuthology.orchestra.run.trial099.stdout:created 2026-02-20T11:42:12.256417+0000 2026-02-20T11:42:12.259 INFO:teuthology.orchestra.run.trial099.stdout:min_mon_release 20 (tentacle) 2026-02-20T11:42:12.259 INFO:teuthology.orchestra.run.trial099.stdout:election_strategy: 1 2026-02-20T11:42:12.259 INFO:teuthology.orchestra.run.trial099.stdout:0: [v2:10.20.193.99:3300/0,v1:10.20.193.99:6789/0] mon.a 2026-02-20T11:42:12.259 INFO:teuthology.orchestra.run.trial099.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-20T11:42:12.260 DEBUG:teuthology.orchestra.run.trial099:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T11:42:12.317 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 90b0dec6-5a72-419e-80f0-d25365c72a06... 2026-02-20T11:42:12.318 DEBUG:teuthology.orchestra.run.trial099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-20T11:42:12.398 INFO:teuthology.orchestra.run.trial099.stdout:[global] 2026-02-20T11:42:12.398 INFO:teuthology.orchestra.run.trial099.stdout: chdir = "" 2026-02-20T11:42:12.398 INFO:teuthology.orchestra.run.trial099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-20T11:42:12.398 INFO:teuthology.orchestra.run.trial099.stdout: auth supported = cephx 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: filestore xattr use omap = true 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: mon clock drift allowed = 1.000 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: osd crush chooseleaf type = 0 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: auth debug = true 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: ms die on old message = true 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: ms die on bug = true 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.399 INFO:teuthology.orchestra.run.trial099.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-20T11:42:12.400 INFO:teuthology.orchestra.run.trial099.stdout: mon pg warn max object skew = 0 2026-02-20T11:42:12.400 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.400 INFO:teuthology.orchestra.run.trial099.stdout: # disable pg_autoscaler by default for new pools 2026-02-20T11:42:12.400 INFO:teuthology.orchestra.run.trial099.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-20T11:42:12.400 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.400 INFO:teuthology.orchestra.run.trial099.stdout: osd pool default size = 2 2026-02-20T11:42:12.400 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.400 INFO:teuthology.orchestra.run.trial099.stdout: mon osd allow primary affinity = true 2026-02-20T11:42:12.400 INFO:teuthology.orchestra.run.trial099.stdout: mon osd allow pg remap = true 2026-02-20T11:42:12.400 INFO:teuthology.orchestra.run.trial099.stdout: mon warn on legacy crush tunables = false 2026-02-20T11:42:12.400 INFO:teuthology.orchestra.run.trial099.stdout: mon warn on crush straw calc version zero = false 2026-02-20T11:42:12.401 INFO:teuthology.orchestra.run.trial099.stdout: mon warn on no sortbitwise = false 2026-02-20T11:42:12.401 INFO:teuthology.orchestra.run.trial099.stdout: mon warn on osd down out interval zero = false 2026-02-20T11:42:12.401 INFO:teuthology.orchestra.run.trial099.stdout: mon warn on too few osds = false 2026-02-20T11:42:12.401 INFO:teuthology.orchestra.run.trial099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-20T11:42:12.401 INFO:teuthology.orchestra.run.trial099.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-20T11:42:12.401 INFO:teuthology.orchestra.run.trial099.stdout: mon_allow_pool_size_one = true 2026-02-20T11:42:12.401 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.401 INFO:teuthology.orchestra.run.trial099.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-20T11:42:12.401 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.401 INFO:teuthology.orchestra.run.trial099.stdout: osd default data pool replay window = 5 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: mon allow pool delete = true 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: mon cluster log file level = debug 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: debug asserts on shutdown = true 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: mon health detail to clog = false 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: mon host = 10.20.193.99 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: ms client mode = crc 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: ms cluster mode = crc 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: ms mon client mode = crc 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: ms mon cluster mode = crc 2026-02-20T11:42:12.402 INFO:teuthology.orchestra.run.trial099.stdout: ms mon service mode = crc 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout: ms service mode = crc 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout: fsid = 90b0dec6-5a72-419e-80f0-d25365c72a06 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout:[osd] 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout: osd journal size = 100 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout: osd scrub load threshold = 5.0 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout: osd scrub max interval = 600 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout: osd mclock profile = high_recovery_ops 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout: osd mclock skip benchmark = true 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.403 INFO:teuthology.orchestra.run.trial099.stdout: osd recover clone overlap = true 2026-02-20T11:42:12.404 INFO:teuthology.orchestra.run.trial099.stdout: osd recovery max chunk = 1048576 2026-02-20T11:42:12.404 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.404 INFO:teuthology.orchestra.run.trial099.stdout: osd debug shutdown = true 2026-02-20T11:42:12.404 INFO:teuthology.orchestra.run.trial099.stdout: osd debug op order = true 2026-02-20T11:42:12.404 INFO:teuthology.orchestra.run.trial099.stdout: osd debug verify stray on activate = true 2026-02-20T11:42:12.404 INFO:teuthology.orchestra.run.trial099.stdout: osd debug trim objects = true 2026-02-20T11:42:12.404 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.404 INFO:teuthology.orchestra.run.trial099.stdout: osd open classes on start = true 2026-02-20T11:42:12.404 INFO:teuthology.orchestra.run.trial099.stdout: osd debug pg log writeout = true 2026-02-20T11:42:12.404 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.404 INFO:teuthology.orchestra.run.trial099.stdout: osd deep scrub update digest min age = 30 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: osd map max advance = 10 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: journal zero on create = true 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: filestore ondisk finisher threads = 3 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: filestore apply finisher threads = 3 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: bdev debug aio = true 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: osd debug misdirected ops = true 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: debug monc = 20 2026-02-20T11:42:12.405 INFO:teuthology.orchestra.run.trial099.stdout: crimson cpu num = 4 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: crimson osd obc lru size = 10 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug ms = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug osd = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore backref = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore cache = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore cleaner = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore device = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore epm = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore fixedkv tree = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore journal = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore lba = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore odata = 20 2026-02-20T11:42:12.406 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore omap = 20 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore onode = 20 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore t = 20 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout: debug seastore tm = 20 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout: osd objectstore = seastore 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout: osd shutdown pgref assert = True 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout: seastore cachepin size pershard = 64M 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout: seastore cachepin type = 2Q 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout: seastore max concurrent transactions = 8 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout:[mgr] 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout: debug ms = 1 2026-02-20T11:42:12.407 INFO:teuthology.orchestra.run.trial099.stdout: debug mgr = 20 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: debug mon = 20 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: debug auth = 20 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: mon reweight min pgs per osd = 4 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: mon reweight min bytes per osd = 10 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: mgr/telemetry/nag = false 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout:[mon] 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: debug ms = 1 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: debug mon = 20 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: debug paxos = 20 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: debug auth = 20 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: mon data avail warn = 5 2026-02-20T11:42:12.408 INFO:teuthology.orchestra.run.trial099.stdout: mon mgr mkfs grace = 240 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: mon reweight min pgs per osd = 4 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: mon osd reporter subtree level = osd 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: mon osd prime pg temp = true 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: mon reweight min bytes per osd = 10 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: auth mon ticket ttl = 660 # 11m 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: auth service ticket ttl = 240 # 4m 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: # don't complain about insecure global_id in the test suite 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-20T11:42:12.409 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout: # 1m isn't quite enough 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout: mon_down_mkfs_grace = 2m 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout: mon_warn_on_filestore_osds = false 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout: mon min osdmap epochs = 50 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout: mon osdmap full prune interval = 2 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout: mon osdmap full prune min = 15 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout: mon osdmap full prune txsize = 2 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout: paxos service trim min = 10 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout: osd pool default crimson = True 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:12.410 INFO:teuthology.orchestra.run.trial099.stdout:[client] 2026-02-20T11:42:12.411 INFO:teuthology.orchestra.run.trial099.stdout: rgw cache enabled = true 2026-02-20T11:42:12.411 INFO:teuthology.orchestra.run.trial099.stdout: rgw enable ops log = true 2026-02-20T11:42:12.411 INFO:teuthology.orchestra.run.trial099.stdout: rgw enable usage log = true 2026-02-20T11:42:12.411 INFO:teuthology.orchestra.run.trial099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-20T11:42:12.411 INFO:teuthology.orchestra.run.trial099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-20T11:42:12.411 INFO:teuthology.orchestra.run.trial099.stdout:[mon.a] 2026-02-20T11:42:12.412 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-20T11:42:12.413 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-20T11:42:12.504 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-20T11:42:12.504 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:12.505 DEBUG:teuthology.orchestra.run.trial099:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-20T11:42:12.522 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:12.522 DEBUG:teuthology.orchestra.run.trial099:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-20T11:42:12.581 INFO:tasks.ceph:Sending monmap to node ubuntu@trial099.front.sepia.ceph.com 2026-02-20T11:42:12.581 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:12.581 DEBUG:teuthology.orchestra.run.trial099:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-20T11:42:12.582 DEBUG:teuthology.orchestra.run.trial099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T11:42:12.655 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:12.655 DEBUG:teuthology.orchestra.run.trial099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:42:12.714 INFO:tasks.ceph:Setting up mon nodes... 2026-02-20T11:42:12.714 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-20T11:42:12.714 DEBUG:teuthology.orchestra.run.trial099:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T11:42:12.805 INFO:teuthology.orchestra.run.trial099.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T11:42:12.808 INFO:tasks.ceph:Setting up mds nodes... 2026-02-20T11:42:12.808 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-20T11:42:12.808 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-20T11:42:12.849 INFO:teuthology.orchestra.run.trial099.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-20T11:42:12.861 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-20T11:42:12.862 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T11:42:12.862 DEBUG:teuthology.orchestra.run.trial099:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-20T11:42:12.926 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:42:12.926 INFO:tasks.ceph:role: osd.0 2026-02-20T11:42:12.926 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial099.front.sepia.ceph.com 2026-02-20T11:42:12.926 DEBUG:teuthology.orchestra.run.trial099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-20T11:42:12.991 INFO:teuthology.orchestra.run.trial099.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:42:12.992 INFO:teuthology.orchestra.run.trial099.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:42:12.992 INFO:teuthology.orchestra.run.trial099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:42:12.992 INFO:teuthology.orchestra.run.trial099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:42:12.992 INFO:teuthology.orchestra.run.trial099.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:42:12.992 INFO:teuthology.orchestra.run.trial099.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:42:12.992 INFO:teuthology.orchestra.run.trial099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:42:12.992 INFO:teuthology.orchestra.run.trial099.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:42:12.992 INFO:teuthology.orchestra.run.trial099.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:42:12.992 INFO:teuthology.orchestra.run.trial099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:42:13.573 INFO:teuthology.orchestra.run.trial099.stdout:Discarding blocks...Done. 2026-02-20T11:42:13.575 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial099.front.sepia.ceph.com -o noatime 2026-02-20T11:42:13.575 DEBUG:teuthology.orchestra.run.trial099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-20T11:42:13.643 DEBUG:teuthology.orchestra.run.trial099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-20T11:42:13.711 DEBUG:teuthology.orchestra.run.trial099:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-20T11:42:13.774 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:42:13.775 INFO:tasks.ceph:role: osd.1 2026-02-20T11:42:13.775 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial099.front.sepia.ceph.com 2026-02-20T11:42:13.775 DEBUG:teuthology.orchestra.run.trial099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-20T11:42:13.839 INFO:teuthology.orchestra.run.trial099.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:42:13.840 INFO:teuthology.orchestra.run.trial099.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:42:13.840 INFO:teuthology.orchestra.run.trial099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:42:13.840 INFO:teuthology.orchestra.run.trial099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:42:13.840 INFO:teuthology.orchestra.run.trial099.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:42:13.840 INFO:teuthology.orchestra.run.trial099.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:42:13.840 INFO:teuthology.orchestra.run.trial099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:42:13.840 INFO:teuthology.orchestra.run.trial099.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:42:13.840 INFO:teuthology.orchestra.run.trial099.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:42:13.840 INFO:teuthology.orchestra.run.trial099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:42:14.402 INFO:teuthology.orchestra.run.trial099.stdout:Discarding blocks...Done. 2026-02-20T11:42:14.404 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial099.front.sepia.ceph.com -o noatime 2026-02-20T11:42:14.404 DEBUG:teuthology.orchestra.run.trial099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-20T11:42:14.473 DEBUG:teuthology.orchestra.run.trial099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-20T11:42:14.541 DEBUG:teuthology.orchestra.run.trial099:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-20T11:42:14.607 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:42:14.607 INFO:tasks.ceph:role: osd.2 2026-02-20T11:42:14.607 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial099.front.sepia.ceph.com 2026-02-20T11:42:14.608 DEBUG:teuthology.orchestra.run.trial099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-20T11:42:14.672 INFO:teuthology.orchestra.run.trial099.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:42:14.673 INFO:teuthology.orchestra.run.trial099.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:42:14.673 INFO:teuthology.orchestra.run.trial099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:42:14.673 INFO:teuthology.orchestra.run.trial099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:42:14.673 INFO:teuthology.orchestra.run.trial099.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:42:14.673 INFO:teuthology.orchestra.run.trial099.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:42:14.673 INFO:teuthology.orchestra.run.trial099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:42:14.673 INFO:teuthology.orchestra.run.trial099.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:42:14.673 INFO:teuthology.orchestra.run.trial099.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:42:14.673 INFO:teuthology.orchestra.run.trial099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:42:15.237 INFO:teuthology.orchestra.run.trial099.stdout:Discarding blocks...Done. 2026-02-20T11:42:15.239 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial099.front.sepia.ceph.com -o noatime 2026-02-20T11:42:15.239 DEBUG:teuthology.orchestra.run.trial099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-20T11:42:15.308 DEBUG:teuthology.orchestra.run.trial099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-20T11:42:15.375 DEBUG:teuthology.orchestra.run.trial099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:42:15.527 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:42:15.580 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:15,579 seastar - Reactor backend: linux-aio 2026-02-20T11:42:15.589 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:15,588 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:15.589 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:15,588 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:15.591 INFO:teuthology.orchestra.run.trial099.stderr:==39307==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:42:15.647 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:15,645 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T11:42:15.647 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:15,646 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T11:42:15.647 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:15,646 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T11:42:15.647 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:15,646 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T11:42:15.648 INFO:teuthology.orchestra.run.trial099.stderr:DEBUG 2026-02-20 11:42:15,646 [shard 0:main] seastar - reactor::drain 2026-02-20T11:42:17.702 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:17.703 INFO:teuthology.orchestra.run.trial099.stderr:Suppressions used: 2026-02-20T11:42:17.703 INFO:teuthology.orchestra.run.trial099.stderr: count bytes template 2026-02-20T11:42:17.703 INFO:teuthology.orchestra.run.trial099.stderr: 1 8 MallocExtension::Register 2026-02-20T11:42:17.703 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:17.703 INFO:teuthology.orchestra.run.trial099.stderr: 2026-02-20T11:42:17.721 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:42:17.722 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:42:17.766 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:42:17.766 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:42:17.773 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:17,772 seastar - Reactor backend: linux-aio 2026-02-20T11:42:17.784 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:17,783 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:17.784 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:17,783 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:17.788 INFO:teuthology.orchestra.run.trial099.stderr:==39305==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:42:17.788 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:17,787 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:42:17.845 INFO:teuthology.orchestra.run.trial099.stderr:DEBUG 2026-02-20 11:42:17,844 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T11:42:17.927 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9f1d48d 0x9f1d69f 0x9f1d868 0x9f1d8fb 0x9f1e740 0x78bcf19 0x7b9c686 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:17.927 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0xcc3279 0x10a37e0 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e20729 0x9e20c76 0x9e20e75 0x9f0d69d 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:17.927 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c1b /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9cf0b5b 0x9d70916 0x9d70a2c 0x9d70b08 0x9d70d84 0x9d7132b 0x9d7172f 0x9d719ec 0x9f0638f 0x9f0656d 0x9f067c9 0x9f068ba 0x9f0745a 0x9f07a5e 0x9f07df1 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:17.959 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xfd19a /lib64/libasan.so.8+0xec7e9 0xcc3105 0x9e6544d 0x9eee572 0x9eee75a 0x9eee806 0x9f17355 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:42:18.002 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191ccd /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x78327ca 0x9552b79 0x959870c 0x96246d0 0x90ebcea 0x90ec1c0 0x9167135 0x7a42cf7 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:18.002 INFO:teuthology.orchestra.run.trial099.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa4ae98a6 0xffffffffa47cb921 0xffffffffa47d3aeb 0xffffffffa47d3f4b 0xffffffffa44910d2 0xffffffffa514d451 0xffffffffa5200ba2 2026-02-20T11:42:18.002 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c32 /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9ef014c 0x9ef025d 0x7b8b13a 0x7ba0607 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:18.031 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15277 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:42:18.066 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:18.068 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:18.069 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d8872 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:18.193 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0xcc3231 0x1138f42 0xf4864f 0x9f0887d 0x9f08bc2 0x9f08f45 0x9f1909c 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:18.193 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe419a1 0x78bcd67 0x7b9c686 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:18.193 INFO:teuthology.orchestra.run.trial099.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa4494300 0xffffffffa461f7c7 0xffffffffa461fb20 0xffffffffa449525e 0xffffffffa47e61a9 0xffffffffa47c9715 0xffffffffa47d3aeb 0xffffffffa47d3f4b 0xffffffffa44910d2 0xffffffffa514d451 0xffffffffa5200ba2 2026-02-20T11:42:18.193 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc589 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe419a1 0x9efa612 0x9efa7ce 0x9efa937 0x9efaa98 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:18.194 INFO:teuthology.orchestra.run.trial099.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa4ac5c91 0xffffffffa47f9163 0xffffffffa47f9774 0xffffffffa47fb709 0xffffffffa47fcc31 0xffffffffa4824893 0xffffffffa44933eb 0xffffffffa47cb6f7 0xffffffffa47cbad5 0xffffffffa47d3aeb 0xffffffffa47d3f4b 0xffffffffa44910d2 0xffffffffa514d451 0xffffffffa5200ba2 2026-02-20T11:42:18.276 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d7d88 0x7d69b1d 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:42:18.291 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d6 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:18.291 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d7d87 0x7d69b1d 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:18.291 INFO:teuthology.orchestra.run.trial099.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa4f0838d 0xffffffffa4f08ef6 0xffffffffa4f09745 0xffffffffa4f099fb 0xffffffffc0a0ffe8 0xffffffffa4f09b59 0xffffffffa4f0a254 0xffffffffa451e57d 0xffffffffa451e851 0xffffffffa514a7a3 0xffffffffa5200da2 2026-02-20T11:42:18.292 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d834d 0x7d69b2a 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:20.458 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:20.458 INFO:teuthology.orchestra.run.trial099.stderr:Suppressions used: 2026-02-20T11:42:20.458 INFO:teuthology.orchestra.run.trial099.stderr: count bytes template 2026-02-20T11:42:20.458 INFO:teuthology.orchestra.run.trial099.stderr: 1 8 MallocExtension::Register 2026-02-20T11:42:20.458 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:20.458 INFO:teuthology.orchestra.run.trial099.stderr: 2026-02-20T11:42:20.495 DEBUG:teuthology.orchestra.run.trial099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-20T11:42:20.520 DEBUG:teuthology.orchestra.run.trial099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:42:20.675 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:42:20.732 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:20,731 seastar - Reactor backend: linux-aio 2026-02-20T11:42:20.743 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:20,742 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:20.743 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:20,742 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:20.745 INFO:teuthology.orchestra.run.trial099.stderr:==39372==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:42:20.800 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:20,799 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T11:42:20.800 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:20,799 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T11:42:20.800 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:20,799 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T11:42:20.801 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:20,799 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T11:42:20.801 INFO:teuthology.orchestra.run.trial099.stderr:DEBUG 2026-02-20 11:42:20,799 [shard 0:main] seastar - reactor::drain 2026-02-20T11:42:22.844 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:22.844 INFO:teuthology.orchestra.run.trial099.stderr:Suppressions used: 2026-02-20T11:42:22.844 INFO:teuthology.orchestra.run.trial099.stderr: count bytes template 2026-02-20T11:42:22.844 INFO:teuthology.orchestra.run.trial099.stderr: 1 8 MallocExtension::Register 2026-02-20T11:42:22.844 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:22.845 INFO:teuthology.orchestra.run.trial099.stderr: 2026-02-20T11:42:22.864 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:42:22.864 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:42:22.905 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:42:22.905 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:42:22.912 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:22,911 seastar - Reactor backend: linux-aio 2026-02-20T11:42:22.926 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:22,925 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:22.926 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:22,925 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:22.930 INFO:teuthology.orchestra.run.trial099.stderr:==39370==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:42:22.930 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:22,930 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:42:22.987 INFO:teuthology.orchestra.run.trial099.stderr:DEBUG 2026-02-20 11:42:22,987 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T11:42:23.068 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0xcc3237 0x10a3600 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e20729 0x9e20c76 0x9e20e75 0x9f0d69d 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.069 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x1138fc8 0xf4864f 0x9e1e492 0x9e1e78f 0x9e1ea02 0x9e1ecb8 0x9e1f27a 0x9e27721 0x9e27e01 0x9f17e54 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.069 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x359d335 0x9e30705 0x9e307c7 0x9ef7d44 0x9ef7ed9 0x9ef826b 0x9ef9851 0x9efab6c 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.101 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0xff18e /lib64/libasan.so.8+0x470c3 /lib64/libstdc++.so.6+0xd0f6d /lib64/libstdc++.so.6+0x1138d1 /lib64/libstdc++.so.6+0x113d17 /lib64/libstdc++.so.6+0x111a9c /lib64/libstdc++.so.6+0x139f62 0xa81b720 0x95d1578 0x96243fb 0x90ebcea 0x90ec1c0 0x9167135 0x7a42cf7 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:42:23.101 INFO:teuthology.orchestra.run.trial099.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa514d8c6 0xffffffffa5148f66 0xffffffffa520012b 2026-02-20T11:42:23.144 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1588f 0x3549a64 0x3549be4 0x359d2e4 0x9e30705 0x9e307e8 0x9ef7d44 0x9ef7ed9 0x9ef826b 0x9ef9851 0x9efab6c 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7f67185 0x7fb45b4 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.144 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfd239 /lib64/libasan.so.8+0xee859 0xbcac15 0xbcae1e 0xbcb35b 0xd08566 0x1852eee 0x9efabe7 0x9efaf8c 0x9f19778 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7f67185 0x7fb45b4 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.144 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191cf5 /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x1367182 0x4b4f421 0x4accea5 0x7e5a182 0x7fb3f68 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.144 INFO:teuthology.orchestra.run.trial099.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa4b0ba89 0xffffffffa47d0d37 0xffffffffa44910a4 0xffffffffa514d451 0xffffffffa5200ba2 2026-02-20T11:42:23.168 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:42:23.209 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xf4e8f /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.209 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.210 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x297f447 0x29d89ca 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.334 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xe942 /lib64/libasan.so.8+0xec776 0xcc3105 0x9efac13 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.335 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b36a /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xb7b689 0x10a36ac 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e1ecb8 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.335 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9e1e776 0x9e1ea02 0x9e1fa02 0x9e1ff70 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.417 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:42:23.433 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d89b2 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.434 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d89b2 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:23.434 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151c1 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:25.641 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:25.641 INFO:teuthology.orchestra.run.trial099.stderr:Suppressions used: 2026-02-20T11:42:25.641 INFO:teuthology.orchestra.run.trial099.stderr: count bytes template 2026-02-20T11:42:25.642 INFO:teuthology.orchestra.run.trial099.stderr: 1 8 MallocExtension::Register 2026-02-20T11:42:25.642 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:25.642 INFO:teuthology.orchestra.run.trial099.stderr: 2026-02-20T11:42:25.678 DEBUG:teuthology.orchestra.run.trial099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-20T11:42:25.701 DEBUG:teuthology.orchestra.run.trial099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:42:25.858 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:42:25.922 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:25,921 seastar - Reactor backend: linux-aio 2026-02-20T11:42:25.934 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:25,933 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:25.934 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:25,933 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:25.936 INFO:teuthology.orchestra.run.trial099.stderr:==39440==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:42:25.992 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:25,990 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T11:42:25.992 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:25,990 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T11:42:25.992 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:25,990 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T11:42:25.992 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:25,990 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T11:42:25.992 INFO:teuthology.orchestra.run.trial099.stderr:DEBUG 2026-02-20 11:42:25,991 [shard 0:main] seastar - reactor::drain 2026-02-20T11:42:28.075 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:28.075 INFO:teuthology.orchestra.run.trial099.stderr:Suppressions used: 2026-02-20T11:42:28.075 INFO:teuthology.orchestra.run.trial099.stderr: count bytes template 2026-02-20T11:42:28.075 INFO:teuthology.orchestra.run.trial099.stderr: 1 8 MallocExtension::Register 2026-02-20T11:42:28.075 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:28.075 INFO:teuthology.orchestra.run.trial099.stderr: 2026-02-20T11:42:28.095 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:42:28.095 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:42:28.136 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:42:28.136 INFO:teuthology.orchestra.run.trial099.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:42:28.143 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:28,142 seastar - Reactor backend: linux-aio 2026-02-20T11:42:28.157 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:28,156 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:28.157 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:28,156 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:42:28.161 INFO:teuthology.orchestra.run.trial099.stderr:==39438==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:42:28.161 INFO:teuthology.orchestra.run.trial099.stderr:INFO 2026-02-20 11:42:28,160 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:42:28.223 INFO:teuthology.orchestra.run.trial099.stderr:DEBUG 2026-02-20 11:42:28,222 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T11:42:28.305 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15ef5 0x9e1e650 0x9e1ea02 0x9e1ecb8 0x9e1f27a 0x9e27721 0x9e27e01 0x9f17e54 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.305 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c1b /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9cf0b5b 0x9ee0e7e 0x9ee0fac 0x9f137b3 0x9f13f56 0x9f1e423 0x78bcf19 0x7b9c081 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.305 INFO:teuthology.orchestra.run.trial099.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa47cbab0 0xffffffffa47d3aeb 0xffffffffa47d3f4b 0xffffffffa44910d2 0xffffffffa514d451 0xffffffffa5200ba2 2026-02-20T11:42:28.305 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x361f /lib64/libasan.so.8+0x157c7 0xe62790 0x7abd05b 0x7b9cece 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.337 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c6 /lib64/libasan.so.8+0x15197 0x1138fc8 0xf4864f 0x9f0887d 0x9f08bc2 0x9f08f45 0x9f1909c 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b8bc1a 0x7ba02df 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:42:28.380 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9f1b381 0x9edc6d1 0x9edb6d1 0x7f67185 0x7fb45b4 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.380 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15eaf 0x9f07ef2 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7f67185 0x7fb45b4 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.380 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfcaf3 /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x157c7 0x9e209f2 0x9e20e75 0x9f0d69d 0x9f0ddf8 0x9f19c81 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x911c2c7 0x911d4dc 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.406 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129747 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:42:28.446 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0f /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.446 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d6 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.447 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1517f 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.574 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ed3 /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x1138fc8 0xf4864f 0x9f085d5 0x9f08bc2 0x9f08f45 0x9f1909c 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.574 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b364 /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xb7b689 0x10a36ac 0xf4865a 0x9e1e492 0x9e1e78f 0x9e1ea02 0x9e1ecb8 0x9e1f27a 0x9e27721 0x9e27e01 0x9f17e54 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.574 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x158a9 0x359e59f 0x9ef8230 0x9ef9851 0x9efab6c 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.656 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d79ec 0x7d69b08 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:42:28.672 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d7d63 0x7d69b1d 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.672 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129859 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:28.673 INFO:teuthology.orchestra.run.trial099.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d7811 0x7d69b08 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:42:30.830 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:30.830 INFO:teuthology.orchestra.run.trial099.stderr:Suppressions used: 2026-02-20T11:42:30.830 INFO:teuthology.orchestra.run.trial099.stderr: count bytes template 2026-02-20T11:42:30.831 INFO:teuthology.orchestra.run.trial099.stderr: 1 8 MallocExtension::Register 2026-02-20T11:42:30.831 INFO:teuthology.orchestra.run.trial099.stderr:----------------------------------------------------- 2026-02-20T11:42:30.831 INFO:teuthology.orchestra.run.trial099.stderr: 2026-02-20T11:42:30.865 DEBUG:teuthology.orchestra.run.trial099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-20T11:42:30.891 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-20T11:42:30.892 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:30.892 DEBUG:teuthology.orchestra.run.trial099:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-20T11:42:30.956 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:30.957 DEBUG:teuthology.orchestra.run.trial099:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-20T11:42:31.021 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:31.021 DEBUG:teuthology.orchestra.run.trial099:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-20T11:42:31.086 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:31.086 DEBUG:teuthology.orchestra.run.trial099:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-20T11:42:31.149 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:31.150 DEBUG:teuthology.orchestra.run.trial099:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-20T11:42:31.206 INFO:tasks.ceph:Adding keys to all mons... 2026-02-20T11:42:31.206 DEBUG:teuthology.orchestra.run.trial099:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-20T11:42:31.267 INFO:teuthology.orchestra.run.trial099.stdout:[mgr.x] 2026-02-20T11:42:31.268 INFO:teuthology.orchestra.run.trial099.stdout: key = AQCUSJhpfMf1LxAAIwpJJcl82jdEv7/5Oe371Q== 2026-02-20T11:42:31.268 INFO:teuthology.orchestra.run.trial099.stdout:[osd.0] 2026-02-20T11:42:31.268 INFO:teuthology.orchestra.run.trial099.stdout: key = AQCZSJhpSZKRMhAAq3sjbeGguwBxcId7rHXzhA== 2026-02-20T11:42:31.268 INFO:teuthology.orchestra.run.trial099.stdout:[osd.1] 2026-02-20T11:42:31.268 INFO:teuthology.orchestra.run.trial099.stdout: key = AQCeSJhpRRYPOxAAsL/gquGFcxWJvvqLMIDuCg== 2026-02-20T11:42:31.268 INFO:teuthology.orchestra.run.trial099.stdout:[osd.2] 2026-02-20T11:42:31.268 INFO:teuthology.orchestra.run.trial099.stdout: key = AQCkSJhp1P6CDRAAnlAf40Ru2Wm4eum+CXFnmg== 2026-02-20T11:42:31.268 INFO:teuthology.orchestra.run.trial099.stdout:[client.0] 2026-02-20T11:42:31.268 INFO:teuthology.orchestra.run.trial099.stdout: key = AQCUSJhp2PCeMhAA/CPSQJHqzrXnVnZLO401Dg== 2026-02-20T11:42:31.270 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-20T11:42:31.358 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:42:31.407 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:42:31.455 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:42:31.505 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-20T11:42:31.555 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-20T11:42:31.555 DEBUG:teuthology.orchestra.run.trial099:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-20T11:42:31.581 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-20T11:42:31.669 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:31.667+0000 7fa9c4262c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:31.669 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:31.667+0000 7fa9c4262c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:31.681 DEBUG:teuthology.orchestra.run.trial099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-20T11:42:31.706 DEBUG:teuthology.orchestra.run.trial099:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:42:31.761 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-20T11:42:31.762 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-20T11:42:31.762 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-20T11:42:31.804 INFO:tasks.ceph.mon.a:Started 2026-02-20T11:42:31.804 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-20T11:42:31.804 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-20T11:42:31.804 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-20T11:42:31.807 INFO:tasks.ceph.mgr.x:Started 2026-02-20T11:42:31.807 DEBUG:tasks.ceph:set 0 configs 2026-02-20T11:42:31.807 DEBUG:teuthology.orchestra.run.trial099:> sudo ceph --cluster ceph config dump 2026-02-20T11:42:31.865 INFO:tasks.ceph.mon.a.trial099.stderr:2026-02-20T11:42:31.863+0000 7fe660062c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:31.865 INFO:tasks.ceph.mon.a.trial099.stderr:2026-02-20T11:42:31.863+0000 7fe660062c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:31.866 INFO:tasks.ceph.mon.a.trial099.stderr:2026-02-20T11:42:31.863+0000 7fe660062c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:31.870 INFO:tasks.ceph.mgr.x.trial099.stderr:2026-02-20T11:42:31.868+0000 7f2f013c9fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:31.881 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:31.878+0000 7ffa596cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:31.885 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:31.883+0000 7ffa596cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:31.917 INFO:teuthology.orchestra.run.trial099.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-20T11:42:31.925 INFO:tasks.ceph:Setting crush tunables to default 2026-02-20T11:42:31.926 DEBUG:teuthology.orchestra.run.trial099:> sudo ceph --cluster ceph osd crush tunables default 2026-02-20T11:42:32.000 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:31.995+0000 7f2a1ce8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:32.003 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:32.001+0000 7f2a17577640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:32.037 INFO:teuthology.orchestra.run.trial099.stderr:adjusted tunables profile to default 2026-02-20T11:42:32.045 INFO:tasks.ceph:check_enable_crimson: True 2026-02-20T11:42:32.045 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-20T11:42:32.045 DEBUG:teuthology.orchestra.run.trial099:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-20T11:42:32.073 INFO:tasks.ceph.mgr.x.trial099.stderr:2026-02-20T11:42:32.072+0000 7f2f013c9fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:32.074 INFO:tasks.ceph.mgr.x.trial099.stderr:2026-02-20T11:42:32.072+0000 7f2f013c9fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:32.118 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:32.113+0000 7f40a1164640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:32.120 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:32.119+0000 7f40a1164640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:32.160 INFO:tasks.ceph.mon.a.trial099.stderr:2026-02-20T11:42:32.158+0000 7fe6583f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-20T11:42:32.160 INFO:tasks.ceph.mon.a.trial099.stderr:Please be aware that this feature is experimental, untested, 2026-02-20T11:42:32.160 INFO:tasks.ceph.mon.a.trial099.stderr:unsupported, and may result in data corruption, data loss, 2026-02-20T11:42:32.160 INFO:tasks.ceph.mon.a.trial099.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-20T11:42:32.160 INFO:tasks.ceph.mon.a.trial099.stderr:feature with important data. 2026-02-20T11:42:32.160 INFO:tasks.ceph.mon.a.trial099.stderr: 2026-02-20T11:42:32.914 INFO:tasks.ceph.mgr.x.trial099.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-20T11:42:32.914 INFO:tasks.ceph.mgr.x.trial099.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-20T11:42:32.914 INFO:tasks.ceph.mgr.x.trial099.stderr: from numpy import show_config as show_numpy_config 2026-02-20T11:42:33.048 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-20T11:42:33.049 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:33.049 DEBUG:teuthology.orchestra.run.trial099:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-20T11:42:33.113 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:33.113 DEBUG:teuthology.orchestra.run.trial099:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-20T11:42:33.176 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:42:33.176 DEBUG:teuthology.orchestra.run.trial099:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-20T11:42:33.243 DEBUG:teuthology.orchestra.run.trial099:> sudo ceph --cluster ceph osd new 300e4291-6ae3-4d3d-88f5-2a9945ed5ea2 0 2026-02-20T11:42:33.357 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:33.353+0000 7f998170b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:33.359 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:33.358+0000 7f998170b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:33.397 INFO:teuthology.orchestra.run.trial099.stdout:0 2026-02-20T11:42:33.405 DEBUG:teuthology.orchestra.run.trial099:> sudo ceph --cluster ceph osd new f414ad41-95b1-493c-9601-29ddab00f8f8 1 2026-02-20T11:42:33.478 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:33.473+0000 7f7041e3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:33.480 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:33.479+0000 7f7041e3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:33.519 INFO:teuthology.orchestra.run.trial099.stdout:1 2026-02-20T11:42:33.528 DEBUG:teuthology.orchestra.run.trial099:> sudo ceph --cluster ceph osd new 1540233a-5e9a-4429-931e-507e6ba08549 2 2026-02-20T11:42:33.604 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:33.598+0000 7f62bd8cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:33.606 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:33.605+0000 7f62bd8cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:33.644 INFO:teuthology.orchestra.run.trial099.stdout:2 2026-02-20T11:42:33.653 DEBUG:teuthology.orchestra.run.trial099:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-20T11:42:33.681 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-20T11:42:33.681 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-20T11:42:33.723 INFO:tasks.ceph.osd.0:Started 2026-02-20T11:42:33.723 DEBUG:teuthology.orchestra.run.trial099:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-20T11:42:33.752 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-20T11:42:33.752 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-20T11:42:33.795 INFO:tasks.ceph.osd.1:Started 2026-02-20T11:42:33.795 DEBUG:teuthology.orchestra.run.trial099:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-20T11:42:33.822 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-20T11:42:33.823 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-20T11:42:33.866 INFO:tasks.ceph.osd.2:Started 2026-02-20T11:42:33.866 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:42:33.944 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:33.940+0000 7f046e4b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:33.946 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:33.945+0000 7f046e4b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:33.981 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:33.981 INFO:teuthology.orchestra.run.trial099.stdout:{"epoch":6,"fsid":"90b0dec6-5a72-419e-80f0-d25365c72a06","created":"2026-02-20T11:42:31.876982+0000","modified":"2026-02-20T11:42:33.643517+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:42:33.643517+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"300e4291-6ae3-4d3d-88f5-2a9945ed5ea2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f414ad41-95b1-493c-9601-29ddab00f8f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1540233a-5e9a-4429-931e-507e6ba08549","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:42:33.991 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-20T11:42:33.991 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-20T11:42:34.293 DEBUG:teuthology.orchestra.run.trial099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:42:34.362 INFO:teuthology.misc.health.trial099.stderr:2026-02-20T11:42:34.358+0000 7f7500bec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:34.365 INFO:teuthology.misc.health.trial099.stderr:2026-02-20T11:42:34.363+0000 7f7500bec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:34.398 INFO:teuthology.misc.health.trial099.stdout: 2026-02-20T11:42:34.398 INFO:teuthology.misc.health.trial099.stdout:{"epoch":6,"fsid":"90b0dec6-5a72-419e-80f0-d25365c72a06","created":"2026-02-20T11:42:31.876982+0000","modified":"2026-02-20T11:42:33.643517+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:42:33.643517+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"300e4291-6ae3-4d3d-88f5-2a9945ed5ea2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f414ad41-95b1-493c-9601-29ddab00f8f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1540233a-5e9a-4429-931e-507e6ba08549","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:42:34.406 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T11:42:35.301 INFO:tasks.ceph.mgr.x.trial099.stderr:2026-02-20T11:42:35.299+0000 7f2e1c2f8640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-20T11:42:40.708 DEBUG:teuthology.orchestra.run.trial099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:42:40.805 INFO:teuthology.misc.health.trial099.stderr:2026-02-20T11:42:40.801+0000 7f45e29b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:40.810 INFO:teuthology.misc.health.trial099.stderr:2026-02-20T11:42:40.808+0000 7f45e29b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:40.944 INFO:teuthology.misc.health.trial099.stdout: 2026-02-20T11:42:40.944 INFO:teuthology.misc.health.trial099.stdout:{"epoch":7,"fsid":"90b0dec6-5a72-419e-80f0-d25365c72a06","created":"2026-02-20T11:42:31.876982+0000","modified":"2026-02-20T11:42:40.437748+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:42:33.643517+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"300e4291-6ae3-4d3d-88f5-2a9945ed5ea2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f414ad41-95b1-493c-9601-29ddab00f8f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1540233a-5e9a-4429-931e-507e6ba08549","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:42:40.958 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T11:42:47.260 DEBUG:teuthology.orchestra.run.trial099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:42:47.330 INFO:teuthology.misc.health.trial099.stderr:2026-02-20T11:42:47.326+0000 7fb4e6f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:47.332 INFO:teuthology.misc.health.trial099.stderr:2026-02-20T11:42:47.330+0000 7fb4e6f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:47.449 INFO:teuthology.misc.health.trial099.stdout: 2026-02-20T11:42:47.449 INFO:teuthology.misc.health.trial099.stdout:{"epoch":13,"fsid":"90b0dec6-5a72-419e-80f0-d25365c72a06","created":"2026-02-20T11:42:31.876982+0000","modified":"2026-02-20T11:42:46.464829+0000","last_up_change":"2026-02-20T11:42:43.451672+0000","last_in_change":"2026-02-20T11:42:33.643517+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:42:44.310333+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"300e4291-6ae3-4d3d-88f5-2a9945ed5ea2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6805","nonce":264855339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6804","nonce":264855339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6819","nonce":264855339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6818","nonce":264855339}]},"public_addr":"10.20.193.99:6805/264855339","cluster_addr":"10.20.193.99:6804/264855339","heartbeat_back_addr":"10.20.193.99:6819/264855339","heartbeat_front_addr":"10.20.193.99:6818/264855339","state":["exists","up"]},{"osd":1,"uuid":"f414ad41-95b1-493c-9601-29ddab00f8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6807","nonce":206149368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6806","nonce":206149368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6810","nonce":206149368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6811","nonce":206149368}]},"public_addr":"10.20.193.99:6807/206149368","cluster_addr":"10.20.193.99:6806/206149368","heartbeat_back_addr":"10.20.193.99:6810/206149368","heartbeat_front_addr":"10.20.193.99:6811/206149368","state":["exists","up"]},{"osd":2,"uuid":"1540233a-5e9a-4429-931e-507e6ba08549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6802","nonce":1090531805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6803","nonce":1090531805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6812","nonce":1090531805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6813","nonce":1090531805}]},"public_addr":"10.20.193.99:6802/1090531805","cluster_addr":"10.20.193.99:6803/1090531805","heartbeat_back_addr":"10.20.193.99:6812/1090531805","heartbeat_front_addr":"10.20.193.99:6813/1090531805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:42:47.457 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-20T11:42:47.457 INFO:tasks.ceph:Creating RBD pool 2026-02-20T11:42:47.457 DEBUG:teuthology.orchestra.run.trial099:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-20T11:42:47.532 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:47.527+0000 7f5d4facb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:47.534 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:47.533+0000 7f5d4facb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:48.474 INFO:teuthology.orchestra.run.trial099.stderr:pool 'rbd' created 2026-02-20T11:42:48.485 DEBUG:teuthology.orchestra.run.trial099:> rbd --cluster ceph pool init rbd 2026-02-20T11:42:48.524 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:48.523+0000 7f97fa417240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:48.524 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:48.523+0000 7f97fa417240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:48.528 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:48.527+0000 7f97fa417240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:51.511 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-20T11:42:51.513 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-20T11:42:51.513 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-20T11:42:51.628 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:51.624+0000 7fd569b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:51.630 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:51.629+0000 7fd569b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:51.742 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:51.751 INFO:teuthology.orchestra.run.trial099.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-20T11:42:51.751 INFO:tasks.ceph_manager:config epoch is 1 2026-02-20T11:42:51.752 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-20T11:42:51.752 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-20T11:42:51.752 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-20T11:42:51.827 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:51.823+0000 7f7dd88cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:51.829 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:51.827+0000 7f7dd88cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:51.974 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:51.981 INFO:teuthology.orchestra.run.trial099.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6800","nonce":640726229},{"type":"v1","addr":"10.20.193.99:6801","nonce":640726229}]},"active_addr":"10.20.193.99:6801/640726229","active_change":"2026-02-20T11:42:34.294481+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.99:0","nonce":2287290790}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.99:0","nonce":1465574400}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.99:0","nonce":2827826350}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.99:0","nonce":1712182029}]}]} 2026-02-20T11:42:51.986 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-20T11:42:51.987 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-20T11:42:51.987 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:42:52.064 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.061+0000 7f4ef5f9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.066 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.065+0000 7f4ef5f9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.179 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:52.179 INFO:teuthology.orchestra.run.trial099.stdout:{"epoch":17,"fsid":"90b0dec6-5a72-419e-80f0-d25365c72a06","created":"2026-02-20T11:42:31.876982+0000","modified":"2026-02-20T11:42:51.482375+0000","last_up_change":"2026-02-20T11:42:43.451672+0000","last_in_change":"2026-02-20T11:42:33.643517+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:42:44.310333+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T11:42:47.651131+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"300e4291-6ae3-4d3d-88f5-2a9945ed5ea2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6805","nonce":264855339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6804","nonce":264855339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6819","nonce":264855339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6818","nonce":264855339}]},"public_addr":"10.20.193.99:6805/264855339","cluster_addr":"10.20.193.99:6804/264855339","heartbeat_back_addr":"10.20.193.99:6819/264855339","heartbeat_front_addr":"10.20.193.99:6818/264855339","state":["exists","up"]},{"osd":1,"uuid":"f414ad41-95b1-493c-9601-29ddab00f8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6807","nonce":206149368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6806","nonce":206149368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6810","nonce":206149368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6811","nonce":206149368}]},"public_addr":"10.20.193.99:6807/206149368","cluster_addr":"10.20.193.99:6806/206149368","heartbeat_back_addr":"10.20.193.99:6810/206149368","heartbeat_front_addr":"10.20.193.99:6811/206149368","state":["exists","up"]},{"osd":2,"uuid":"1540233a-5e9a-4429-931e-507e6ba08549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6802","nonce":1090531805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6803","nonce":1090531805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6812","nonce":1090531805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6813","nonce":1090531805}]},"public_addr":"10.20.193.99:6802/1090531805","cluster_addr":"10.20.193.99:6803/1090531805","heartbeat_back_addr":"10.20.193.99:6812/1090531805","heartbeat_front_addr":"10.20.193.99:6813/1090531805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:42:52.189 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-20T11:42:52.189 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:42:52.266 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.262+0000 7f7f18d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.268 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.266+0000 7f7f18d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.381 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:52.381 INFO:teuthology.orchestra.run.trial099.stdout:{"epoch":17,"fsid":"90b0dec6-5a72-419e-80f0-d25365c72a06","created":"2026-02-20T11:42:31.876982+0000","modified":"2026-02-20T11:42:51.482375+0000","last_up_change":"2026-02-20T11:42:43.451672+0000","last_in_change":"2026-02-20T11:42:33.643517+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:42:44.310333+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T11:42:47.651131+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"300e4291-6ae3-4d3d-88f5-2a9945ed5ea2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6805","nonce":264855339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6804","nonce":264855339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6819","nonce":264855339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6818","nonce":264855339}]},"public_addr":"10.20.193.99:6805/264855339","cluster_addr":"10.20.193.99:6804/264855339","heartbeat_back_addr":"10.20.193.99:6819/264855339","heartbeat_front_addr":"10.20.193.99:6818/264855339","state":["exists","up"]},{"osd":1,"uuid":"f414ad41-95b1-493c-9601-29ddab00f8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6807","nonce":206149368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6806","nonce":206149368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6810","nonce":206149368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6811","nonce":206149368}]},"public_addr":"10.20.193.99:6807/206149368","cluster_addr":"10.20.193.99:6806/206149368","heartbeat_back_addr":"10.20.193.99:6810/206149368","heartbeat_front_addr":"10.20.193.99:6811/206149368","state":["exists","up"]},{"osd":2,"uuid":"1540233a-5e9a-4429-931e-507e6ba08549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6802","nonce":1090531805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6803","nonce":1090531805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6812","nonce":1090531805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6813","nonce":1090531805}]},"public_addr":"10.20.193.99:6802/1090531805","cluster_addr":"10.20.193.99:6803/1090531805","heartbeat_back_addr":"10.20.193.99:6812/1090531805","heartbeat_front_addr":"10.20.193.99:6813/1090531805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:42:52.391 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-20T11:42:52.392 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-20T11:42:52.392 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-20T11:42:52.472 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.468+0000 7f4761e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.473 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.468+0000 7f6046a1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.473 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.469+0000 7f4f2c10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.475 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.473+0000 7f4761e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.475 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.474+0000 7f6046a1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.476 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.474+0000 7f4f2c10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.493 INFO:teuthology.orchestra.run.trial099.stdout:42949672969 2026-02-20T11:42:52.493 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T11:42:52.494 INFO:teuthology.orchestra.run.trial099.stdout:38654705674 2026-02-20T11:42:52.495 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T11:42:52.496 INFO:teuthology.orchestra.run.trial099.stdout:42949672969 2026-02-20T11:42:52.496 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T11:42:52.614 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.610+0000 7f782e2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.615 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.610+0000 7f391f411640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.615 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.611+0000 7f9926606640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.617 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.616+0000 7f782e2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.618 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.616+0000 7f391f411640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.618 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:52.616+0000 7f9926606640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:52.738 INFO:teuthology.orchestra.run.trial099.stdout:42949672964 2026-02-20T11:42:52.739 INFO:teuthology.orchestra.run.trial099.stdout:42949672964 2026-02-20T11:42:52.744 INFO:teuthology.orchestra.run.trial099.stdout:38654705673 2026-02-20T11:42:52.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672964 for osd.1 2026-02-20T11:42:52.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672964 for osd.0 2026-02-20T11:42:52.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705673 for osd.2 2026-02-20T11:42:53.750 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T11:42:53.751 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T11:42:53.755 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T11:42:53.830 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:53.825+0000 7efd8bb0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:53.832 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:53.828+0000 7f0c0310b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:53.832 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:53.831+0000 7efd8bb0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:53.835 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:53.833+0000 7f0c0310b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:53.873 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:53.868+0000 7f4506ba4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:53.875 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:53.873+0000 7f4506ba4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:53.947 INFO:teuthology.orchestra.run.trial099.stdout:42949672964 2026-02-20T11:42:53.951 INFO:teuthology.orchestra.run.trial099.stdout:42949672964 2026-02-20T11:42:53.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672964 for osd.1 2026-02-20T11:42:53.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672964 for osd.0 2026-02-20T11:42:53.990 INFO:teuthology.orchestra.run.trial099.stdout:38654705673 2026-02-20T11:42:54.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705673 for osd.2 2026-02-20T11:42:54.959 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T11:42:54.964 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T11:42:55.001 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T11:42:55.038 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.033+0000 7ffb3725a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.041 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.039+0000 7ffb3725a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.081 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.077+0000 7f8e0c15c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.082 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.077+0000 7f695e10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.084 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.082+0000 7f695e10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.084 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.082+0000 7f8e0c15c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.157 INFO:teuthology.orchestra.run.trial099.stdout:42949672969 2026-02-20T11:42:55.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.1 2026-02-20T11:42:55.169 DEBUG:teuthology.parallel:result is None 2026-02-20T11:42:55.202 INFO:teuthology.orchestra.run.trial099.stdout:42949672969 2026-02-20T11:42:55.204 INFO:teuthology.orchestra.run.trial099.stdout:38654705674 2026-02-20T11:42:55.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.0 2026-02-20T11:42:55.213 DEBUG:teuthology.parallel:result is None 2026-02-20T11:42:55.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705674 for osd.2 2026-02-20T11:42:55.216 DEBUG:teuthology.parallel:result is None 2026-02-20T11:42:55.216 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-20T11:42:55.216 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:42:55.333 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.329+0000 7f47d24cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.335 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.333+0000 7f47d24cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.447 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:55.448 INFO:teuthology.orchestra.run.trial099.stderr:dumped all 2026-02-20T11:42:55.456 INFO:teuthology.orchestra.run.trial099.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-20T11:42:54.298833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2836,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497132,"statfs":{"total":160255967232,"available":160253063168,"internally_reserved":0,"allocated":2904064,"data_stored":2904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.825387"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.494618+0000","last_change":"2026-02-20T11:42:51.498308+0000","last_active":"2026-02-20T11:42:51.494618+0000","last_peered":"2026-02-20T11:42:51.494618+0000","last_clean":"2026-02-20T11:42:51.494618+0000","last_became_active":"2026-02-20T11:42:49.514790+0000","last_became_peered":"2026-02-20T11:42:49.514790+0000","last_unstale":"2026-02-20T11:42:51.494618+0000","last_undegraded":"2026-02-20T11:42:51.494618+0000","last_fullsized":"2026-02-20T11:42:51.494618+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.496311+0000","last_change":"2026-02-20T11:42:51.499716+0000","last_active":"2026-02-20T11:42:51.496311+0000","last_peered":"2026-02-20T11:42:51.496311+0000","last_clean":"2026-02-20T11:42:51.496311+0000","last_became_active":"2026-02-20T11:42:49.514448+0000","last_became_peered":"2026-02-20T11:42:49.514448+0000","last_unstale":"2026-02-20T11:42:51.496311+0000","last_undegraded":"2026-02-20T11:42:51.496311+0000","last_fullsized":"2026-02-20T11:42:51.496311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.498835+0000","last_change":"2026-02-20T11:42:51.502396+0000","last_active":"2026-02-20T11:42:51.498835+0000","last_peered":"2026-02-20T11:42:51.498835+0000","last_clean":"2026-02-20T11:42:51.498835+0000","last_became_active":"2026-02-20T11:42:49.519266+0000","last_became_peered":"2026-02-20T11:42:49.519266+0000","last_unstale":"2026-02-20T11:42:51.498835+0000","last_undegraded":"2026-02-20T11:42:51.498835+0000","last_fullsized":"2026-02-20T11:42:51.498835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.496316+0000","last_change":"2026-02-20T11:42:51.499384+0000","last_active":"2026-02-20T11:42:51.496316+0000","last_peered":"2026-02-20T11:42:51.496316+0000","last_clean":"2026-02-20T11:42:51.496316+0000","last_became_active":"2026-02-20T11:42:49.518650+0000","last_became_peered":"2026-02-20T11:42:49.518650+0000","last_unstale":"2026-02-20T11:42:51.496316+0000","last_undegraded":"2026-02-20T11:42:51.496316+0000","last_fullsized":"2026-02-20T11:42:51.496316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.499119+0000","last_change":"2026-02-20T11:42:51.500262+0000","last_active":"2026-02-20T11:42:51.499119+0000","last_peered":"2026-02-20T11:42:51.499119+0000","last_clean":"2026-02-20T11:42:51.499119+0000","last_became_active":"2026-02-20T11:42:49.529853+0000","last_became_peered":"2026-02-20T11:42:49.529853+0000","last_unstale":"2026-02-20T11:42:51.499119+0000","last_undegraded":"2026-02-20T11:42:51.499119+0000","last_fullsized":"2026-02-20T11:42:51.499119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.496578+0000","last_change":"2026-02-20T11:42:51.499330+0000","last_active":"2026-02-20T11:42:51.496578+0000","last_peered":"2026-02-20T11:42:51.496578+0000","last_clean":"2026-02-20T11:42:51.496578+0000","last_became_active":"2026-02-20T11:42:49.531735+0000","last_became_peered":"2026-02-20T11:42:49.531735+0000","last_unstale":"2026-02-20T11:42:51.496578+0000","last_undegraded":"2026-02-20T11:42:51.496578+0000","last_fullsized":"2026-02-20T11:42:51.496578+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.494248+0000","last_change":"2026-02-20T11:42:51.496871+0000","last_active":"2026-02-20T11:42:51.494248+0000","last_peered":"2026-02-20T11:42:51.494248+0000","last_clean":"2026-02-20T11:42:51.494248+0000","last_became_active":"2026-02-20T11:42:49.536183+0000","last_became_peered":"2026-02-20T11:42:49.536183+0000","last_unstale":"2026-02-20T11:42:51.494248+0000","last_undegraded":"2026-02-20T11:42:51.494248+0000","last_fullsized":"2026-02-20T11:42:51.494248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.497509+0000","last_change":"2026-02-20T11:42:51.502354+0000","last_active":"2026-02-20T11:42:51.497509+0000","last_peered":"2026-02-20T11:42:51.497509+0000","last_clean":"2026-02-20T11:42:51.497509+0000","last_became_active":"2026-02-20T11:42:49.511262+0000","last_became_peered":"2026-02-20T11:42:49.511262+0000","last_unstale":"2026-02-20T11:42:51.497509+0000","last_undegraded":"2026-02-20T11:42:51.497509+0000","last_fullsized":"2026-02-20T11:42:51.497509+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.494951+0000","last_change":"2026-02-20T11:42:45.500854+0000","last_active":"2026-02-20T11:42:51.494951+0000","last_peered":"2026-02-20T11:42:51.494951+0000","last_clean":"2026-02-20T11:42:51.494951+0000","last_became_active":"2026-02-20T11:42:45.491808+0000","last_became_peered":"2026-02-20T11:42:45.491808+0000","last_unstale":"2026-02-20T11:42:51.494951+0000","last_undegraded":"2026-02-20T11:42:51.494951+0000","last_fullsized":"2026-02-20T11:42:51.494951+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_clean_scrub_stamp":"2026-02-20T11:42:44.456945+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253063168,"internally_reserved":0,"allocated":2904064,"data_stored":2904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":364,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166292,"statfs":{"total":53418655744,"available":53418283008,"internally_reserved":0,"allocated":372736,"data_stored":372736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1284,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165372,"statfs":{"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1188,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165468,"statfs":{"total":53418655744,"available":53417439232,"internally_reserved":0,"allocated":1216512,"data_stored":1216512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417439232,"internally_reserved":0,"allocated":1216512,"data_stored":1216512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418283008,"internally_reserved":0,"allocated":372736,"data_stored":372736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:42:55.458 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:42:55.535 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.531+0000 7fd326e35640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.537 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.535+0000 7fd326e35640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.649 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:55.650 INFO:teuthology.orchestra.run.trial099.stderr:dumped all 2026-02-20T11:42:55.658 INFO:teuthology.orchestra.run.trial099.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-20T11:42:54.298833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2836,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497132,"statfs":{"total":160255967232,"available":160253063168,"internally_reserved":0,"allocated":2904064,"data_stored":2904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.825387"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.494618+0000","last_change":"2026-02-20T11:42:51.498308+0000","last_active":"2026-02-20T11:42:51.494618+0000","last_peered":"2026-02-20T11:42:51.494618+0000","last_clean":"2026-02-20T11:42:51.494618+0000","last_became_active":"2026-02-20T11:42:49.514790+0000","last_became_peered":"2026-02-20T11:42:49.514790+0000","last_unstale":"2026-02-20T11:42:51.494618+0000","last_undegraded":"2026-02-20T11:42:51.494618+0000","last_fullsized":"2026-02-20T11:42:51.494618+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.496311+0000","last_change":"2026-02-20T11:42:51.499716+0000","last_active":"2026-02-20T11:42:51.496311+0000","last_peered":"2026-02-20T11:42:51.496311+0000","last_clean":"2026-02-20T11:42:51.496311+0000","last_became_active":"2026-02-20T11:42:49.514448+0000","last_became_peered":"2026-02-20T11:42:49.514448+0000","last_unstale":"2026-02-20T11:42:51.496311+0000","last_undegraded":"2026-02-20T11:42:51.496311+0000","last_fullsized":"2026-02-20T11:42:51.496311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.498835+0000","last_change":"2026-02-20T11:42:51.502396+0000","last_active":"2026-02-20T11:42:51.498835+0000","last_peered":"2026-02-20T11:42:51.498835+0000","last_clean":"2026-02-20T11:42:51.498835+0000","last_became_active":"2026-02-20T11:42:49.519266+0000","last_became_peered":"2026-02-20T11:42:49.519266+0000","last_unstale":"2026-02-20T11:42:51.498835+0000","last_undegraded":"2026-02-20T11:42:51.498835+0000","last_fullsized":"2026-02-20T11:42:51.498835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.496316+0000","last_change":"2026-02-20T11:42:51.499384+0000","last_active":"2026-02-20T11:42:51.496316+0000","last_peered":"2026-02-20T11:42:51.496316+0000","last_clean":"2026-02-20T11:42:51.496316+0000","last_became_active":"2026-02-20T11:42:49.518650+0000","last_became_peered":"2026-02-20T11:42:49.518650+0000","last_unstale":"2026-02-20T11:42:51.496316+0000","last_undegraded":"2026-02-20T11:42:51.496316+0000","last_fullsized":"2026-02-20T11:42:51.496316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.499119+0000","last_change":"2026-02-20T11:42:51.500262+0000","last_active":"2026-02-20T11:42:51.499119+0000","last_peered":"2026-02-20T11:42:51.499119+0000","last_clean":"2026-02-20T11:42:51.499119+0000","last_became_active":"2026-02-20T11:42:49.529853+0000","last_became_peered":"2026-02-20T11:42:49.529853+0000","last_unstale":"2026-02-20T11:42:51.499119+0000","last_undegraded":"2026-02-20T11:42:51.499119+0000","last_fullsized":"2026-02-20T11:42:51.499119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.496578+0000","last_change":"2026-02-20T11:42:51.499330+0000","last_active":"2026-02-20T11:42:51.496578+0000","last_peered":"2026-02-20T11:42:51.496578+0000","last_clean":"2026-02-20T11:42:51.496578+0000","last_became_active":"2026-02-20T11:42:49.531735+0000","last_became_peered":"2026-02-20T11:42:49.531735+0000","last_unstale":"2026-02-20T11:42:51.496578+0000","last_undegraded":"2026-02-20T11:42:51.496578+0000","last_fullsized":"2026-02-20T11:42:51.496578+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.494248+0000","last_change":"2026-02-20T11:42:51.496871+0000","last_active":"2026-02-20T11:42:51.494248+0000","last_peered":"2026-02-20T11:42:51.494248+0000","last_clean":"2026-02-20T11:42:51.494248+0000","last_became_active":"2026-02-20T11:42:49.536183+0000","last_became_peered":"2026-02-20T11:42:49.536183+0000","last_unstale":"2026-02-20T11:42:51.494248+0000","last_undegraded":"2026-02-20T11:42:51.494248+0000","last_fullsized":"2026-02-20T11:42:51.494248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.497509+0000","last_change":"2026-02-20T11:42:51.502354+0000","last_active":"2026-02-20T11:42:51.497509+0000","last_peered":"2026-02-20T11:42:51.497509+0000","last_clean":"2026-02-20T11:42:51.497509+0000","last_became_active":"2026-02-20T11:42:49.511262+0000","last_became_peered":"2026-02-20T11:42:49.511262+0000","last_unstale":"2026-02-20T11:42:51.497509+0000","last_undegraded":"2026-02-20T11:42:51.497509+0000","last_fullsized":"2026-02-20T11:42:51.497509+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T11:42:51.494951+0000","last_change":"2026-02-20T11:42:45.500854+0000","last_active":"2026-02-20T11:42:51.494951+0000","last_peered":"2026-02-20T11:42:51.494951+0000","last_clean":"2026-02-20T11:42:51.494951+0000","last_became_active":"2026-02-20T11:42:45.491808+0000","last_became_peered":"2026-02-20T11:42:45.491808+0000","last_unstale":"2026-02-20T11:42:51.494951+0000","last_undegraded":"2026-02-20T11:42:51.494951+0000","last_fullsized":"2026-02-20T11:42:51.494951+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_clean_scrub_stamp":"2026-02-20T11:42:44.456945+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253063168,"internally_reserved":0,"allocated":2904064,"data_stored":2904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":364,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166292,"statfs":{"total":53418655744,"available":53418283008,"internally_reserved":0,"allocated":372736,"data_stored":372736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1284,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165372,"statfs":{"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1188,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165468,"statfs":{"total":53418655744,"available":53417439232,"internally_reserved":0,"allocated":1216512,"data_stored":1216512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417439232,"internally_reserved":0,"allocated":1216512,"data_stored":1216512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418283008,"internally_reserved":0,"allocated":372736,"data_stored":372736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:42:55.660 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-20T11:42:55.660 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-20T11:42:55.660 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-20T11:42:55.660 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-20T11:42:55.736 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.733+0000 7f5b5d0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.738 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:42:55.736+0000 7f5b5d0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:55.881 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:42:55.881 INFO:teuthology.orchestra.run.trial099.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-20T11:42:55.891 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-20T11:42:55.891 INFO:teuthology.run_tasks:Running task workunit... 2026-02-20T11:42:55.917 INFO:tasks.workunit:Pulling workunits from ref 445d5c9ba18a4040e70e25303230c36c77773acb 2026-02-20T11:42:55.918 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-20T11:42:55.918 DEBUG:teuthology.orchestra.run.trial099:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-20T11:42:55.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:42:55.936 INFO:teuthology.orchestra.run.trial099.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-20T11:42:55.936 DEBUG:teuthology.orchestra.run.trial099:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-20T11:42:55.993 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-20T11:42:55.993 DEBUG:teuthology.orchestra.run.trial099:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-20T11:42:56.049 INFO:tasks.workunit:timeout=3h 2026-02-20T11:42:56.049 INFO:tasks.workunit:cleanup=True 2026-02-20T11:42:56.050 DEBUG:teuthology.orchestra.run.trial099:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 445d5c9ba18a4040e70e25303230c36c77773acb 2026-02-20T11:42:56.112 INFO:tasks.workunit.client.0.trial099.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-20T11:43:18.334 INFO:tasks.workunit.client.0.trial099.stderr:Note: switching to '445d5c9ba18a4040e70e25303230c36c77773acb'. 2026-02-20T11:43:18.335 INFO:tasks.workunit.client.0.trial099.stderr: 2026-02-20T11:43:18.335 INFO:tasks.workunit.client.0.trial099.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-20T11:43:18.335 INFO:tasks.workunit.client.0.trial099.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-20T11:43:18.335 INFO:tasks.workunit.client.0.trial099.stderr:state without impacting any branches by switching back to a branch. 2026-02-20T11:43:18.335 INFO:tasks.workunit.client.0.trial099.stderr: 2026-02-20T11:43:18.335 INFO:tasks.workunit.client.0.trial099.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-20T11:43:18.335 INFO:tasks.workunit.client.0.trial099.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-20T11:43:18.335 INFO:tasks.workunit.client.0.trial099.stderr: 2026-02-20T11:43:18.335 INFO:tasks.workunit.client.0.trial099.stderr: git switch -c 2026-02-20T11:43:18.335 INFO:tasks.workunit.client.0.trial099.stderr: 2026-02-20T11:43:18.336 INFO:tasks.workunit.client.0.trial099.stderr:Or undo this operation with: 2026-02-20T11:43:18.336 INFO:tasks.workunit.client.0.trial099.stderr: 2026-02-20T11:43:18.336 INFO:tasks.workunit.client.0.trial099.stderr: git switch - 2026-02-20T11:43:18.336 INFO:tasks.workunit.client.0.trial099.stderr: 2026-02-20T11:43:18.336 INFO:tasks.workunit.client.0.trial099.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-20T11:43:18.336 INFO:tasks.workunit.client.0.trial099.stderr: 2026-02-20T11:43:18.336 INFO:tasks.workunit.client.0.trial099.stderr:HEAD is now at 445d5c9ba18 Merge PR #66907 into main 2026-02-20T11:43:18.337 DEBUG:teuthology.orchestra.run.trial099:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-20T11:43:18.394 INFO:tasks.workunit.client.0.trial099.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-20T11:43:18.396 INFO:tasks.workunit.client.0.trial099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-20T11:43:18.396 INFO:tasks.workunit.client.0.trial099.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-20T11:43:18.486 INFO:tasks.workunit.client.0.trial099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-20T11:43:18.517 INFO:tasks.workunit.client.0.trial099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-20T11:43:18.543 INFO:tasks.workunit.client.0.trial099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-20T11:43:18.544 INFO:tasks.workunit.client.0.trial099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-20T11:43:18.544 INFO:tasks.workunit.client.0.trial099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-20T11:43:18.569 INFO:tasks.workunit.client.0.trial099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-20T11:43:18.572 DEBUG:teuthology.orchestra.run.trial099:> set -ex 2026-02-20T11:43:18.572 DEBUG:teuthology.orchestra.run.trial099:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-20T11:43:18.630 INFO:tasks.workunit:Running workunits matching rbd/test_lock_fence.sh on client.0... 2026-02-20T11:43:18.632 INFO:tasks.workunit:Running workunit rbd/test_lock_fence.sh... 2026-02-20T11:43:18.633 DEBUG:teuthology.orchestra.run.trial099:workunit test rbd/test_lock_fence.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=445d5c9ba18a4040e70e25303230c36c77773acb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_lock_fence.sh 2026-02-20T11:43:18.690 INFO:tasks.workunit.client.0.trial099.stderr:+ IMAGE=rbdrw-image 2026-02-20T11:43:18.690 INFO:tasks.workunit.client.0.trial099.stderr:+ LOCKID=rbdrw 2026-02-20T11:43:18.690 INFO:tasks.workunit.client.0.trial099.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_lock_fence.sh 2026-02-20T11:43:18.691 INFO:tasks.workunit.client.0.trial099.stderr:+ RELPATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/librbd 2026-02-20T11:43:18.691 INFO:tasks.workunit.client.0.trial099.stderr:+ RBDRW=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/librbd/rbdrw.py 2026-02-20T11:43:18.691 INFO:tasks.workunit.client.0.trial099.stderr:+ rbd create rbdrw-image --size 10 --image-format 2 --image-shared 2026-02-20T11:43:18.718 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:18.716+0000 7f9548085240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:18.718 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:18.716+0000 7f9548085240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:18.721 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:18.719+0000 7f9548085240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:18.778 INFO:tasks.workunit.client.0.trial099.stderr:+ iochild=42152 2026-02-20T11:43:18.778 INFO:tasks.workunit.client.0.trial099.stderr:+ LOCKS='[]' 2026-02-20T11:43:18.778 INFO:tasks.workunit.client.0.trial099.stderr:+ '[' '[]' == '[]' ']' 2026-02-20T11:43:18.778 INFO:tasks.workunit.client.0.trial099.stderr:+ python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/librbd/rbdrw.py rbdrw-image rbdrw 2026-02-20T11:43:18.778 INFO:tasks.workunit.client.0.trial099.stderr:++ rbd lock list rbdrw-image --format json 2026-02-20T11:43:18.804 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:18.803+0000 7f75c4a4d240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:18.805 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:18.803+0000 7f75c4a4d240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:18.807 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:18.806+0000 7f75c4a4d240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:18.883 INFO:tasks.workunit.client.0.trial099.stderr:+ LOCKS='[]' 2026-02-20T11:43:18.883 INFO:tasks.workunit.client.0.trial099.stderr:+ sleep 1 2026-02-20T11:43:19.884 INFO:tasks.workunit.client.0.trial099.stderr:+ '[' '[]' == '[]' ']' 2026-02-20T11:43:19.884 INFO:tasks.workunit.client.0.trial099.stderr:++ rbd lock list rbdrw-image --format json 2026-02-20T11:43:19.911 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:19.910+0000 7f9df3685240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:19.912 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:19.910+0000 7f9df3685240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:19.914 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:19.913+0000 7f9df3685240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:19.976 INFO:tasks.workunit.client.0.trial099.stderr:+ LOCKS='[{"id":"rbdrw","locker":"client.4218","address":"10.20.193.99:0/1089532739"}]' 2026-02-20T11:43:19.976 INFO:tasks.workunit.client.0.trial099.stderr:+ sleep 1 2026-02-20T11:43:20.978 INFO:tasks.workunit.client.0.trial099.stderr:+ '[' '[{"id":"rbdrw","locker":"client.4218","address":"10.20.193.99:0/1089532739"}]' == '[]' ']' 2026-02-20T11:43:20.978 INFO:tasks.workunit.client.0.trial099.stderr:++ rbd lock list rbdrw-image 2026-02-20T11:43:20.978 INFO:tasks.workunit.client.0.trial099.stderr:++ tail -1 2026-02-20T11:43:20.978 INFO:tasks.workunit.client.0.trial099.stderr:++ awk '{print $NF;}' 2026-02-20T11:43:21.005 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:21.003+0000 7f09daee8240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:21.005 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:21.003+0000 7f09daee8240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:21.008 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:21.006+0000 7f09daee8240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:21.074 INFO:tasks.workunit.client.0.trial099.stderr:+ clientaddr=10.20.193.99:0/1089532739 2026-02-20T11:43:21.074 INFO:tasks.workunit.client.0.trial099.stderr:++ rbd lock list rbdrw-image 2026-02-20T11:43:21.074 INFO:tasks.workunit.client.0.trial099.stderr:++ tail -1 2026-02-20T11:43:21.074 INFO:tasks.workunit.client.0.trial099.stderr:++ awk '{print $1;}' 2026-02-20T11:43:21.100 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:21.099+0000 7fbd4cc10240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:21.101 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:21.099+0000 7fbd4cc10240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:21.103 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:21.102+0000 7fbd4cc10240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:21.162 INFO:tasks.workunit.client.0.trial099.stderr:+ clientid=client.4218 2026-02-20T11:43:21.163 INFO:tasks.workunit.client.0.trial099.stdout:clientaddr: 10.20.193.99:0/1089532739 2026-02-20T11:43:21.163 INFO:tasks.workunit.client.0.trial099.stdout:clientid: client.4218 2026-02-20T11:43:21.163 INFO:tasks.workunit.client.0.trial099.stderr:+ echo 'clientaddr: 10.20.193.99:0/1089532739' 2026-02-20T11:43:21.163 INFO:tasks.workunit.client.0.trial099.stderr:+ echo 'clientid: client.4218' 2026-02-20T11:43:21.163 INFO:tasks.workunit.client.0.trial099.stderr:+ ceph osd blocklist add 10.20.193.99:0/1089532739 2026-02-20T11:43:21.221 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:21.215+0000 7f91ed70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:21.223 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:21.221+0000 7f91ed70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:22.514 INFO:tasks.workunit.client.0.trial099.stderr:blocklisting 10.20.193.99:0/1089532739 until 2026-02-20T12:43:21.578545+0000 (3600 sec) 2026-02-20T11:43:22.530 INFO:tasks.workunit.client.0.trial099.stderr:+ wait 42152 2026-02-20T11:43:22.530 INFO:tasks.workunit.client.0.trial099.stderr:+ rbdrw_exitcode=108 2026-02-20T11:43:22.530 INFO:tasks.workunit.client.0.trial099.stderr:+ '[' 108 '!=' 108 ']' 2026-02-20T11:43:22.530 INFO:tasks.workunit.client.0.trial099.stderr:+ echo 'rbdrw stopped with ESHUTDOWN' 2026-02-20T11:43:22.530 INFO:tasks.workunit.client.0.trial099.stdout:rbdrw stopped with ESHUTDOWN 2026-02-20T11:43:22.531 INFO:tasks.workunit.client.0.trial099.stderr:+ set -e 2026-02-20T11:43:22.531 INFO:tasks.workunit.client.0.trial099.stderr:+ ceph osd blocklist rm 10.20.193.99:0/1089532739 2026-02-20T11:43:22.585 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:22.580+0000 7f9048f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:22.587 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:22.586+0000 7f9048f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:23.586 INFO:tasks.workunit.client.0.trial099.stderr:10.20.193.99:0/1089532739 isn't blocklisted 2026-02-20T11:43:23.595 INFO:tasks.workunit.client.0.trial099.stderr:+ rbd lock remove rbdrw-image rbdrw client.4218 2026-02-20T11:43:23.621 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:23.620+0000 7f7b76dc9240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:23.621 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:23.620+0000 7f7b76dc9240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:23.624 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:23.623+0000 7f7b76dc9240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:24.564 INFO:tasks.workunit.client.0.trial099.stderr:+ sleep 30 2026-02-20T11:43:54.565 INFO:tasks.workunit.client.0.trial099.stderr:+ rbd rm rbdrw-image 2026-02-20T11:43:54.592 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:54.591+0000 7f0618d76240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:54.592 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:54.591+0000 7f0618d76240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:54.595 INFO:tasks.workunit.client.0.trial099.stderr:2026-02-20T11:43:54.594+0000 7f0618d76240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:54.906 INFO:tasks.workunit.client.0.trial099.stderr: Removing image: 33% complete... Removing image: 66% complete... Removing image: 100% complete...done. 2026-02-20T11:43:54.909 INFO:tasks.workunit.client.0.trial099.stderr:+ echo OK 2026-02-20T11:43:54.910 INFO:tasks.workunit.client.0.trial099.stdout:OK 2026-02-20T11:43:54.911 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-20T11:43:54.911 DEBUG:teuthology.orchestra.run.trial099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-20T11:43:54.976 INFO:tasks.workunit:Stopping ['rbd/test_lock_fence.sh'] on client.0... 2026-02-20T11:43:54.976 DEBUG:teuthology.orchestra.run.trial099:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-20T11:43:55.361 DEBUG:teuthology.parallel:result is None 2026-02-20T11:43:55.362 DEBUG:teuthology.orchestra.run.trial099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-20T11:43:55.387 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-20T11:43:55.387 DEBUG:teuthology.orchestra.run.trial099:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-20T11:43:55.444 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-20T11:43:55.445 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-20T11:43:55.460 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-20T11:43:55.460 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:43:55.561 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:55.558+0000 7fd134c1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:55.563 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:55.562+0000 7fd134c1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:55.676 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:43:55.676 INFO:teuthology.orchestra.run.trial099.stderr:dumped all 2026-02-20T11:43:55.685 INFO:teuthology.orchestra.run.trial099.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2026-02-20T11:43:54.307459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594499,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":474,"num_read_kb":1029,"num_write":324,"num_write_kb":2063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":3532,"kb_used_data":3532,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156496436,"statfs":{"total":160255967232,"available":160252350464,"internally_reserved":0,"allocated":3616768,"data_stored":3616768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002516"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536918+0000","last_change":"2026-02-20T11:42:51.498308+0000","last_active":"2026-02-20T11:43:24.536918+0000","last_peered":"2026-02-20T11:43:24.536918+0000","last_clean":"2026-02-20T11:43:24.536918+0000","last_became_active":"2026-02-20T11:42:49.514790+0000","last_became_peered":"2026-02-20T11:42:49.514790+0000","last_unstale":"2026-02-20T11:43:24.536918+0000","last_undegraded":"2026-02-20T11:43:24.536918+0000","last_fullsized":"2026-02-20T11:43:24.536918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537988+0000","last_change":"2026-02-20T11:42:51.499716+0000","last_active":"2026-02-20T11:43:24.537988+0000","last_peered":"2026-02-20T11:43:24.537988+0000","last_clean":"2026-02-20T11:43:24.537988+0000","last_became_active":"2026-02-20T11:42:49.514448+0000","last_became_peered":"2026-02-20T11:42:49.514448+0000","last_unstale":"2026-02-20T11:43:24.537988+0000","last_undegraded":"2026-02-20T11:43:24.537988+0000","last_fullsized":"2026-02-20T11:43:24.537988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"17'1","reported_seq":40,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.538172+0000","last_change":"2026-02-20T11:42:51.502396+0000","last_active":"2026-02-20T11:43:24.538172+0000","last_peered":"2026-02-20T11:43:24.538172+0000","last_clean":"2026-02-20T11:43:24.538172+0000","last_became_active":"2026-02-20T11:42:49.519266+0000","last_became_peered":"2026-02-20T11:42:49.519266+0000","last_unstale":"2026-02-20T11:43:24.538172+0000","last_undegraded":"2026-02-20T11:43:24.538172+0000","last_fullsized":"2026-02-20T11:43:24.538172+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537959+0000","last_change":"2026-02-20T11:42:51.499384+0000","last_active":"2026-02-20T11:43:24.537959+0000","last_peered":"2026-02-20T11:43:24.537959+0000","last_clean":"2026-02-20T11:43:24.537959+0000","last_became_active":"2026-02-20T11:42:49.518650+0000","last_became_peered":"2026-02-20T11:42:49.518650+0000","last_unstale":"2026-02-20T11:43:24.537959+0000","last_undegraded":"2026-02-20T11:43:24.537959+0000","last_fullsized":"2026-02-20T11:43:24.537959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'8","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.551945+0000","last_change":"2026-02-20T11:42:51.500262+0000","last_active":"2026-02-20T11:43:24.551945+0000","last_peered":"2026-02-20T11:43:24.551945+0000","last_clean":"2026-02-20T11:43:24.551945+0000","last_became_active":"2026-02-20T11:42:49.529853+0000","last_became_peered":"2026-02-20T11:42:49.529853+0000","last_unstale":"2026-02-20T11:43:24.551945+0000","last_undegraded":"2026-02-20T11:43:24.551945+0000","last_fullsized":"2026-02-20T11:43:24.551945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":162,"num_read_kb":29,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536647+0000","last_change":"2026-02-20T11:42:51.499330+0000","last_active":"2026-02-20T11:43:24.536647+0000","last_peered":"2026-02-20T11:43:24.536647+0000","last_clean":"2026-02-20T11:43:24.536647+0000","last_became_active":"2026-02-20T11:42:49.531735+0000","last_became_peered":"2026-02-20T11:42:49.531735+0000","last_unstale":"2026-02-20T11:43:24.536647+0000","last_undegraded":"2026-02-20T11:43:24.536647+0000","last_fullsized":"2026-02-20T11:43:24.536647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"17'238","reported_seq":502,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.535742+0000","last_change":"2026-02-20T11:42:51.496871+0000","last_active":"2026-02-20T11:43:24.535742+0000","last_peered":"2026-02-20T11:43:24.535742+0000","last_clean":"2026-02-20T11:43:24.535742+0000","last_became_active":"2026-02-20T11:42:49.536183+0000","last_became_peered":"2026-02-20T11:42:49.536183+0000","last_unstale":"2026-02-20T11:43:24.535742+0000","last_undegraded":"2026-02-20T11:43:24.535742+0000","last_fullsized":"2026-02-20T11:43:24.535742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":237,"num_read_kb":948,"num_write":238,"num_write_kb":952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":28,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537803+0000","last_change":"2026-02-20T11:42:51.502354+0000","last_active":"2026-02-20T11:43:24.537803+0000","last_peered":"2026-02-20T11:43:24.537803+0000","last_clean":"2026-02-20T11:43:24.537803+0000","last_became_active":"2026-02-20T11:42:49.511262+0000","last_became_peered":"2026-02-20T11:42:49.511262+0000","last_unstale":"2026-02-20T11:43:24.537803+0000","last_undegraded":"2026-02-20T11:43:24.537803+0000","last_fullsized":"2026-02-20T11:43:24.537803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536607+0000","last_change":"2026-02-20T11:42:45.500854+0000","last_active":"2026-02-20T11:43:24.536607+0000","last_peered":"2026-02-20T11:43:24.536607+0000","last_clean":"2026-02-20T11:43:24.536607+0000","last_became_active":"2026-02-20T11:42:45.491808+0000","last_became_peered":"2026-02-20T11:42:45.491808+0000","last_unstale":"2026-02-20T11:43:24.536607+0000","last_undegraded":"2026-02-20T11:43:24.536607+0000","last_fullsized":"2026-02-20T11:43:24.536607+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_clean_scrub_stamp":"2026-02-20T11:42:44.456945+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":4131,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":421,"num_read_kb":983,"num_write":245,"num_write_kb":957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160252350464,"internally_reserved":0,"allocated":3616768,"data_stored":3616768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705733,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":488,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166168,"statfs":{"total":53418655744,"available":53418156032,"internally_reserved":0,"allocated":499712,"data_stored":499712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1604,"kb_used_data":1604,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165052,"statfs":{"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1440,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165216,"statfs":{"total":53418655744,"available":53417181184,"internally_reserved":0,"allocated":1474560,"data_stored":1474560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417181184,"internally_reserved":0,"allocated":1474560,"data_stored":1474560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418156032,"internally_reserved":0,"allocated":499712,"data_stored":499712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:43:55.686 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:43:55.761 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:55.758+0000 7f1073ba4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:55.763 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:55.762+0000 7f1073ba4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:55.875 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:43:55.876 INFO:teuthology.orchestra.run.trial099.stderr:dumped all 2026-02-20T11:43:55.884 INFO:teuthology.orchestra.run.trial099.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2026-02-20T11:43:54.307459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594499,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":474,"num_read_kb":1029,"num_write":324,"num_write_kb":2063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":3532,"kb_used_data":3532,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156496436,"statfs":{"total":160255967232,"available":160252350464,"internally_reserved":0,"allocated":3616768,"data_stored":3616768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002516"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536918+0000","last_change":"2026-02-20T11:42:51.498308+0000","last_active":"2026-02-20T11:43:24.536918+0000","last_peered":"2026-02-20T11:43:24.536918+0000","last_clean":"2026-02-20T11:43:24.536918+0000","last_became_active":"2026-02-20T11:42:49.514790+0000","last_became_peered":"2026-02-20T11:42:49.514790+0000","last_unstale":"2026-02-20T11:43:24.536918+0000","last_undegraded":"2026-02-20T11:43:24.536918+0000","last_fullsized":"2026-02-20T11:43:24.536918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537988+0000","last_change":"2026-02-20T11:42:51.499716+0000","last_active":"2026-02-20T11:43:24.537988+0000","last_peered":"2026-02-20T11:43:24.537988+0000","last_clean":"2026-02-20T11:43:24.537988+0000","last_became_active":"2026-02-20T11:42:49.514448+0000","last_became_peered":"2026-02-20T11:42:49.514448+0000","last_unstale":"2026-02-20T11:43:24.537988+0000","last_undegraded":"2026-02-20T11:43:24.537988+0000","last_fullsized":"2026-02-20T11:43:24.537988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"17'1","reported_seq":40,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.538172+0000","last_change":"2026-02-20T11:42:51.502396+0000","last_active":"2026-02-20T11:43:24.538172+0000","last_peered":"2026-02-20T11:43:24.538172+0000","last_clean":"2026-02-20T11:43:24.538172+0000","last_became_active":"2026-02-20T11:42:49.519266+0000","last_became_peered":"2026-02-20T11:42:49.519266+0000","last_unstale":"2026-02-20T11:43:24.538172+0000","last_undegraded":"2026-02-20T11:43:24.538172+0000","last_fullsized":"2026-02-20T11:43:24.538172+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537959+0000","last_change":"2026-02-20T11:42:51.499384+0000","last_active":"2026-02-20T11:43:24.537959+0000","last_peered":"2026-02-20T11:43:24.537959+0000","last_clean":"2026-02-20T11:43:24.537959+0000","last_became_active":"2026-02-20T11:42:49.518650+0000","last_became_peered":"2026-02-20T11:42:49.518650+0000","last_unstale":"2026-02-20T11:43:24.537959+0000","last_undegraded":"2026-02-20T11:43:24.537959+0000","last_fullsized":"2026-02-20T11:43:24.537959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'8","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.551945+0000","last_change":"2026-02-20T11:42:51.500262+0000","last_active":"2026-02-20T11:43:24.551945+0000","last_peered":"2026-02-20T11:43:24.551945+0000","last_clean":"2026-02-20T11:43:24.551945+0000","last_became_active":"2026-02-20T11:42:49.529853+0000","last_became_peered":"2026-02-20T11:42:49.529853+0000","last_unstale":"2026-02-20T11:43:24.551945+0000","last_undegraded":"2026-02-20T11:43:24.551945+0000","last_fullsized":"2026-02-20T11:43:24.551945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":162,"num_read_kb":29,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536647+0000","last_change":"2026-02-20T11:42:51.499330+0000","last_active":"2026-02-20T11:43:24.536647+0000","last_peered":"2026-02-20T11:43:24.536647+0000","last_clean":"2026-02-20T11:43:24.536647+0000","last_became_active":"2026-02-20T11:42:49.531735+0000","last_became_peered":"2026-02-20T11:42:49.531735+0000","last_unstale":"2026-02-20T11:43:24.536647+0000","last_undegraded":"2026-02-20T11:43:24.536647+0000","last_fullsized":"2026-02-20T11:43:24.536647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"17'238","reported_seq":502,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.535742+0000","last_change":"2026-02-20T11:42:51.496871+0000","last_active":"2026-02-20T11:43:24.535742+0000","last_peered":"2026-02-20T11:43:24.535742+0000","last_clean":"2026-02-20T11:43:24.535742+0000","last_became_active":"2026-02-20T11:42:49.536183+0000","last_became_peered":"2026-02-20T11:42:49.536183+0000","last_unstale":"2026-02-20T11:43:24.535742+0000","last_undegraded":"2026-02-20T11:43:24.535742+0000","last_fullsized":"2026-02-20T11:43:24.535742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":237,"num_read_kb":948,"num_write":238,"num_write_kb":952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":28,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537803+0000","last_change":"2026-02-20T11:42:51.502354+0000","last_active":"2026-02-20T11:43:24.537803+0000","last_peered":"2026-02-20T11:43:24.537803+0000","last_clean":"2026-02-20T11:43:24.537803+0000","last_became_active":"2026-02-20T11:42:49.511262+0000","last_became_peered":"2026-02-20T11:42:49.511262+0000","last_unstale":"2026-02-20T11:43:24.537803+0000","last_undegraded":"2026-02-20T11:43:24.537803+0000","last_fullsized":"2026-02-20T11:43:24.537803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536607+0000","last_change":"2026-02-20T11:42:45.500854+0000","last_active":"2026-02-20T11:43:24.536607+0000","last_peered":"2026-02-20T11:43:24.536607+0000","last_clean":"2026-02-20T11:43:24.536607+0000","last_became_active":"2026-02-20T11:42:45.491808+0000","last_became_peered":"2026-02-20T11:42:45.491808+0000","last_unstale":"2026-02-20T11:43:24.536607+0000","last_undegraded":"2026-02-20T11:43:24.536607+0000","last_fullsized":"2026-02-20T11:43:24.536607+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_clean_scrub_stamp":"2026-02-20T11:42:44.456945+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":4131,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":421,"num_read_kb":983,"num_write":245,"num_write_kb":957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160252350464,"internally_reserved":0,"allocated":3616768,"data_stored":3616768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705733,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":488,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166168,"statfs":{"total":53418655744,"available":53418156032,"internally_reserved":0,"allocated":499712,"data_stored":499712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1604,"kb_used_data":1604,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165052,"statfs":{"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1440,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165216,"statfs":{"total":53418655744,"available":53417181184,"internally_reserved":0,"allocated":1474560,"data_stored":1474560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417181184,"internally_reserved":0,"allocated":1474560,"data_stored":1474560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418156032,"internally_reserved":0,"allocated":499712,"data_stored":499712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:43:55.886 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-20T11:43:55.886 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:43:55.967 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:55.963+0000 7fb8fddb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:55.969 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:55.967+0000 7fb8fddb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:56.084 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:43:56.085 INFO:teuthology.orchestra.run.trial099.stderr:dumped all 2026-02-20T11:43:56.094 INFO:teuthology.orchestra.run.trial099.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2026-02-20T11:43:54.307459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594499,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":474,"num_read_kb":1029,"num_write":324,"num_write_kb":2063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":3532,"kb_used_data":3532,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156496436,"statfs":{"total":160255967232,"available":160252350464,"internally_reserved":0,"allocated":3616768,"data_stored":3616768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002516"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536918+0000","last_change":"2026-02-20T11:42:51.498308+0000","last_active":"2026-02-20T11:43:24.536918+0000","last_peered":"2026-02-20T11:43:24.536918+0000","last_clean":"2026-02-20T11:43:24.536918+0000","last_became_active":"2026-02-20T11:42:49.514790+0000","last_became_peered":"2026-02-20T11:42:49.514790+0000","last_unstale":"2026-02-20T11:43:24.536918+0000","last_undegraded":"2026-02-20T11:43:24.536918+0000","last_fullsized":"2026-02-20T11:43:24.536918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537988+0000","last_change":"2026-02-20T11:42:51.499716+0000","last_active":"2026-02-20T11:43:24.537988+0000","last_peered":"2026-02-20T11:43:24.537988+0000","last_clean":"2026-02-20T11:43:24.537988+0000","last_became_active":"2026-02-20T11:42:49.514448+0000","last_became_peered":"2026-02-20T11:42:49.514448+0000","last_unstale":"2026-02-20T11:43:24.537988+0000","last_undegraded":"2026-02-20T11:43:24.537988+0000","last_fullsized":"2026-02-20T11:43:24.537988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"17'1","reported_seq":40,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.538172+0000","last_change":"2026-02-20T11:42:51.502396+0000","last_active":"2026-02-20T11:43:24.538172+0000","last_peered":"2026-02-20T11:43:24.538172+0000","last_clean":"2026-02-20T11:43:24.538172+0000","last_became_active":"2026-02-20T11:42:49.519266+0000","last_became_peered":"2026-02-20T11:42:49.519266+0000","last_unstale":"2026-02-20T11:43:24.538172+0000","last_undegraded":"2026-02-20T11:43:24.538172+0000","last_fullsized":"2026-02-20T11:43:24.538172+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537959+0000","last_change":"2026-02-20T11:42:51.499384+0000","last_active":"2026-02-20T11:43:24.537959+0000","last_peered":"2026-02-20T11:43:24.537959+0000","last_clean":"2026-02-20T11:43:24.537959+0000","last_became_active":"2026-02-20T11:42:49.518650+0000","last_became_peered":"2026-02-20T11:42:49.518650+0000","last_unstale":"2026-02-20T11:43:24.537959+0000","last_undegraded":"2026-02-20T11:43:24.537959+0000","last_fullsized":"2026-02-20T11:43:24.537959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'8","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.551945+0000","last_change":"2026-02-20T11:42:51.500262+0000","last_active":"2026-02-20T11:43:24.551945+0000","last_peered":"2026-02-20T11:43:24.551945+0000","last_clean":"2026-02-20T11:43:24.551945+0000","last_became_active":"2026-02-20T11:42:49.529853+0000","last_became_peered":"2026-02-20T11:42:49.529853+0000","last_unstale":"2026-02-20T11:43:24.551945+0000","last_undegraded":"2026-02-20T11:43:24.551945+0000","last_fullsized":"2026-02-20T11:43:24.551945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":162,"num_read_kb":29,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536647+0000","last_change":"2026-02-20T11:42:51.499330+0000","last_active":"2026-02-20T11:43:24.536647+0000","last_peered":"2026-02-20T11:43:24.536647+0000","last_clean":"2026-02-20T11:43:24.536647+0000","last_became_active":"2026-02-20T11:42:49.531735+0000","last_became_peered":"2026-02-20T11:42:49.531735+0000","last_unstale":"2026-02-20T11:43:24.536647+0000","last_undegraded":"2026-02-20T11:43:24.536647+0000","last_fullsized":"2026-02-20T11:43:24.536647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"17'238","reported_seq":502,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.535742+0000","last_change":"2026-02-20T11:42:51.496871+0000","last_active":"2026-02-20T11:43:24.535742+0000","last_peered":"2026-02-20T11:43:24.535742+0000","last_clean":"2026-02-20T11:43:24.535742+0000","last_became_active":"2026-02-20T11:42:49.536183+0000","last_became_peered":"2026-02-20T11:42:49.536183+0000","last_unstale":"2026-02-20T11:43:24.535742+0000","last_undegraded":"2026-02-20T11:43:24.535742+0000","last_fullsized":"2026-02-20T11:43:24.535742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":237,"num_read_kb":948,"num_write":238,"num_write_kb":952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":28,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537803+0000","last_change":"2026-02-20T11:42:51.502354+0000","last_active":"2026-02-20T11:43:24.537803+0000","last_peered":"2026-02-20T11:43:24.537803+0000","last_clean":"2026-02-20T11:43:24.537803+0000","last_became_active":"2026-02-20T11:42:49.511262+0000","last_became_peered":"2026-02-20T11:42:49.511262+0000","last_unstale":"2026-02-20T11:43:24.537803+0000","last_undegraded":"2026-02-20T11:43:24.537803+0000","last_fullsized":"2026-02-20T11:43:24.537803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536607+0000","last_change":"2026-02-20T11:42:45.500854+0000","last_active":"2026-02-20T11:43:24.536607+0000","last_peered":"2026-02-20T11:43:24.536607+0000","last_clean":"2026-02-20T11:43:24.536607+0000","last_became_active":"2026-02-20T11:42:45.491808+0000","last_became_peered":"2026-02-20T11:42:45.491808+0000","last_unstale":"2026-02-20T11:43:24.536607+0000","last_undegraded":"2026-02-20T11:43:24.536607+0000","last_fullsized":"2026-02-20T11:43:24.536607+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_clean_scrub_stamp":"2026-02-20T11:42:44.456945+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":4131,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":421,"num_read_kb":983,"num_write":245,"num_write_kb":957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160252350464,"internally_reserved":0,"allocated":3616768,"data_stored":3616768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705733,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":488,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166168,"statfs":{"total":53418655744,"available":53418156032,"internally_reserved":0,"allocated":499712,"data_stored":499712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1604,"kb_used_data":1604,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165052,"statfs":{"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1440,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165216,"statfs":{"total":53418655744,"available":53417181184,"internally_reserved":0,"allocated":1474560,"data_stored":1474560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417181184,"internally_reserved":0,"allocated":1474560,"data_stored":1474560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418156032,"internally_reserved":0,"allocated":499712,"data_stored":499712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:43:56.095 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:43:56.172 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:56.168+0000 7ff5bd6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:56.174 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:56.173+0000 7ff5bd6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:56.288 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:43:56.288 INFO:teuthology.orchestra.run.trial099.stdout:{"epoch":21,"fsid":"90b0dec6-5a72-419e-80f0-d25365c72a06","created":"2026-02-20T11:42:31.876982+0000","modified":"2026-02-20T11:43:24.522846+0000","last_up_change":"2026-02-20T11:42:43.451672+0000","last_in_change":"2026-02-20T11:42:33.643517+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:42:44.310333+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T11:42:47.651131+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"300e4291-6ae3-4d3d-88f5-2a9945ed5ea2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6805","nonce":264855339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6804","nonce":264855339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6819","nonce":264855339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6818","nonce":264855339}]},"public_addr":"10.20.193.99:6805/264855339","cluster_addr":"10.20.193.99:6804/264855339","heartbeat_back_addr":"10.20.193.99:6819/264855339","heartbeat_front_addr":"10.20.193.99:6818/264855339","state":["exists","up"]},{"osd":1,"uuid":"f414ad41-95b1-493c-9601-29ddab00f8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6807","nonce":206149368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6806","nonce":206149368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6810","nonce":206149368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6811","nonce":206149368}]},"public_addr":"10.20.193.99:6807/206149368","cluster_addr":"10.20.193.99:6806/206149368","heartbeat_back_addr":"10.20.193.99:6810/206149368","heartbeat_front_addr":"10.20.193.99:6811/206149368","state":["exists","up"]},{"osd":2,"uuid":"1540233a-5e9a-4429-931e-507e6ba08549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6802","nonce":1090531805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6803","nonce":1090531805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6812","nonce":1090531805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.99:6813","nonce":1090531805}]},"public_addr":"10.20.193.99:6802/1090531805","cluster_addr":"10.20.193.99:6803/1090531805","heartbeat_back_addr":"10.20.193.99:6812/1090531805","heartbeat_front_addr":"10.20.193.99:6813/1090531805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.99:0/1089532739":"2026-02-20T12:43:23.686295+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:43:57.299 INFO:tasks.ceph:Scrubbing osd.0 2026-02-20T11:43:57.299 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-20T11:43:57.379 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:57.373+0000 7f0f2765a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:57.381 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:57.380+0000 7f0f2765a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:57.391 INFO:teuthology.orchestra.run.trial099.stdout:{ 2026-02-20T11:43:57.391 INFO:teuthology.orchestra.run.trial099.stdout: "success": "" 2026-02-20T11:43:57.391 INFO:teuthology.orchestra.run.trial099.stdout:} 2026-02-20T11:43:57.398 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-20T11:43:57.475 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:57.470+0000 7fb120c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:57.477 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:57.476+0000 7fb120c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:57.589 INFO:teuthology.orchestra.run.trial099.stderr:instructed osd(s) 0 to deep-scrub 2026-02-20T11:43:57.601 INFO:tasks.ceph:Scrubbing osd.1 2026-02-20T11:43:57.601 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-20T11:43:57.680 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:57.674+0000 7f5e0f011640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:57.682 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:57.681+0000 7f5e0f011640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:57.692 INFO:teuthology.orchestra.run.trial099.stdout:{ 2026-02-20T11:43:57.692 INFO:teuthology.orchestra.run.trial099.stdout: "success": "" 2026-02-20T11:43:57.692 INFO:teuthology.orchestra.run.trial099.stdout:} 2026-02-20T11:43:57.699 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-20T11:43:57.776 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:57.772+0000 7fcab7b5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:57.778 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:57.777+0000 7fcab7b5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:57.890 INFO:teuthology.orchestra.run.trial099.stderr:instructed osd(s) 1 to deep-scrub 2026-02-20T11:43:57.902 INFO:tasks.ceph:Scrubbing osd.2 2026-02-20T11:43:57.902 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-20T11:43:57.982 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:57.976+0000 7f7e8341a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:57.984 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:57.983+0000 7f7e8341a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:57.993 INFO:teuthology.orchestra.run.trial099.stdout:{ 2026-02-20T11:43:57.993 INFO:teuthology.orchestra.run.trial099.stdout: "success": "" 2026-02-20T11:43:57.993 INFO:teuthology.orchestra.run.trial099.stdout:} 2026-02-20T11:43:58.001 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-20T11:43:58.078 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:58.074+0000 7f8a1db0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:58.080 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:58.079+0000 7f8a1db0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:58.193 INFO:teuthology.orchestra.run.trial099.stderr:instructed osd(s) 2 to deep-scrub 2026-02-20T11:43:58.203 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:43:58.280 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:58.276+0000 7f2dee9dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:58.282 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:43:58.280+0000 7f2dee9dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:43:58.395 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:43:58.395 INFO:teuthology.orchestra.run.trial099.stderr:dumped all 2026-02-20T11:43:58.404 INFO:teuthology.orchestra.run.trial099.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2026-02-20T11:43:58.308126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590403,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":474,"num_read_kb":1029,"num_write":325,"num_write_kb":2063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":3592,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156496376,"statfs":{"total":160255967232,"available":160252289024,"internally_reserved":0,"allocated":3678208,"data_stored":3678208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4096,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002350"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536918+0000","last_change":"2026-02-20T11:42:51.498308+0000","last_active":"2026-02-20T11:43:24.536918+0000","last_peered":"2026-02-20T11:43:24.536918+0000","last_clean":"2026-02-20T11:43:24.536918+0000","last_became_active":"2026-02-20T11:42:49.514790+0000","last_became_peered":"2026-02-20T11:42:49.514790+0000","last_unstale":"2026-02-20T11:43:24.536918+0000","last_undegraded":"2026-02-20T11:43:24.536918+0000","last_fullsized":"2026-02-20T11:43:24.536918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537988+0000","last_change":"2026-02-20T11:42:51.499716+0000","last_active":"2026-02-20T11:43:24.537988+0000","last_peered":"2026-02-20T11:43:24.537988+0000","last_clean":"2026-02-20T11:43:24.537988+0000","last_became_active":"2026-02-20T11:42:49.514448+0000","last_became_peered":"2026-02-20T11:42:49.514448+0000","last_unstale":"2026-02-20T11:43:24.537988+0000","last_undegraded":"2026-02-20T11:43:24.537988+0000","last_fullsized":"2026-02-20T11:43:24.537988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"17'1","reported_seq":40,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.538172+0000","last_change":"2026-02-20T11:42:51.502396+0000","last_active":"2026-02-20T11:43:24.538172+0000","last_peered":"2026-02-20T11:43:24.538172+0000","last_clean":"2026-02-20T11:43:24.538172+0000","last_became_active":"2026-02-20T11:42:49.519266+0000","last_became_peered":"2026-02-20T11:42:49.519266+0000","last_unstale":"2026-02-20T11:43:24.538172+0000","last_undegraded":"2026-02-20T11:43:24.538172+0000","last_fullsized":"2026-02-20T11:43:24.538172+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537959+0000","last_change":"2026-02-20T11:42:51.499384+0000","last_active":"2026-02-20T11:43:24.537959+0000","last_peered":"2026-02-20T11:43:24.537959+0000","last_clean":"2026-02-20T11:43:24.537959+0000","last_became_active":"2026-02-20T11:42:49.518650+0000","last_became_peered":"2026-02-20T11:42:49.518650+0000","last_unstale":"2026-02-20T11:43:24.537959+0000","last_undegraded":"2026-02-20T11:43:24.537959+0000","last_fullsized":"2026-02-20T11:43:24.537959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'8","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.551945+0000","last_change":"2026-02-20T11:42:51.500262+0000","last_active":"2026-02-20T11:43:24.551945+0000","last_peered":"2026-02-20T11:43:24.551945+0000","last_clean":"2026-02-20T11:43:24.551945+0000","last_became_active":"2026-02-20T11:42:49.529853+0000","last_became_peered":"2026-02-20T11:42:49.529853+0000","last_unstale":"2026-02-20T11:43:24.551945+0000","last_undegraded":"2026-02-20T11:43:24.551945+0000","last_fullsized":"2026-02-20T11:43:24.551945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":162,"num_read_kb":29,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536647+0000","last_change":"2026-02-20T11:42:51.499330+0000","last_active":"2026-02-20T11:43:24.536647+0000","last_peered":"2026-02-20T11:43:24.536647+0000","last_clean":"2026-02-20T11:43:24.536647+0000","last_became_active":"2026-02-20T11:42:49.531735+0000","last_became_peered":"2026-02-20T11:42:49.531735+0000","last_unstale":"2026-02-20T11:43:24.536647+0000","last_undegraded":"2026-02-20T11:43:24.536647+0000","last_fullsized":"2026-02-20T11:43:24.536647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"17'238","reported_seq":503,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536010+0000","last_change":"2026-02-20T11:42:51.496871+0000","last_active":"2026-02-20T11:43:24.536010+0000","last_peered":"2026-02-20T11:43:24.536010+0000","last_clean":"2026-02-20T11:43:24.536010+0000","last_became_active":"2026-02-20T11:42:49.536183+0000","last_became_peered":"2026-02-20T11:42:49.536183+0000","last_unstale":"2026-02-20T11:43:24.536010+0000","last_undegraded":"2026-02-20T11:43:24.536010+0000","last_fullsized":"2026-02-20T11:43:24.536010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":237,"num_read_kb":948,"num_write":239,"num_write_kb":952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":28,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.537803+0000","last_change":"2026-02-20T11:42:51.502354+0000","last_active":"2026-02-20T11:43:24.537803+0000","last_peered":"2026-02-20T11:43:24.537803+0000","last_clean":"2026-02-20T11:43:24.537803+0000","last_became_active":"2026-02-20T11:42:49.511262+0000","last_became_peered":"2026-02-20T11:42:49.511262+0000","last_unstale":"2026-02-20T11:43:24.537803+0000","last_undegraded":"2026-02-20T11:43:24.537803+0000","last_fullsized":"2026-02-20T11:43:24.537803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:48.470549+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":102,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:24.536607+0000","last_change":"2026-02-20T11:42:45.500854+0000","last_active":"2026-02-20T11:43:24.536607+0000","last_peered":"2026-02-20T11:43:24.536607+0000","last_clean":"2026-02-20T11:43:24.536607+0000","last_became_active":"2026-02-20T11:42:45.491808+0000","last_became_peered":"2026-02-20T11:42:45.491808+0000","last_unstale":"2026-02-20T11:43:24.536607+0000","last_undegraded":"2026-02-20T11:43:24.536607+0000","last_fullsized":"2026-02-20T11:43:24.536607+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:42:44.456945+0000","last_clean_scrub_stamp":"2026-02-20T11:42:44.456945+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":35,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":421,"num_read_kb":983,"num_write":246,"num_write_kb":957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160252289024,"internally_reserved":0,"allocated":3678208,"data_stored":3678208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705738,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":548,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166108,"statfs":{"total":53418655744,"available":53418094592,"internally_reserved":0,"allocated":561152,"data_stored":561152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1604,"kb_used_data":1604,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165052,"statfs":{"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1440,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165216,"statfs":{"total":53418655744,"available":53417181184,"internally_reserved":0,"allocated":1474560,"data_stored":1474560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417181184,"internally_reserved":0,"allocated":1474560,"data_stored":1474560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417013248,"internally_reserved":0,"allocated":1642496,"data_stored":1642496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418094592,"internally_reserved":0,"allocated":561152,"data_stored":561152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:43:58.410 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-20T11:42:48.470549+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=42, tm_sec=48, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=43, tm_sec=56, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:43:58.410 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-20T11:42:48.470549+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=42, tm_sec=48, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=43, tm_sec=56, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:43:58.410 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-20T11:42:48.470549+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=42, tm_sec=48, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=43, tm_sec=56, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:43:58.410 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-20T11:42:48.470549+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=42, tm_sec=48, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=43, tm_sec=56, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:43:58.411 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-20T11:42:48.470549+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=42, tm_sec=48, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=43, tm_sec=56, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:43:58.411 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-20T11:42:48.470549+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=42, tm_sec=48, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=43, tm_sec=56, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:43:58.411 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-20T11:42:48.470549+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=42, tm_sec=48, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=43, tm_sec=56, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:43:58.411 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-20T11:42:48.470549+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=42, tm_sec=48, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=43, tm_sec=56, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:43:58.411 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-20T11:42:44.456945+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=42, tm_sec=44, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=43, tm_sec=56, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:43:58.411 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-20T11:44:18.413 DEBUG:teuthology.orchestra.run.trial099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:44:18.494 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:44:18.490+0000 7f17c2e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:44:18.496 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:44:18.495+0000 7f17c2e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:44:18.609 INFO:teuthology.orchestra.run.trial099.stdout: 2026-02-20T11:44:18.610 INFO:teuthology.orchestra.run.trial099.stderr:dumped all 2026-02-20T11:44:18.619 INFO:teuthology.orchestra.run.trial099.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2026-02-20T11:44:18.311592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":559,"num_read_kb":1041,"num_write":330,"num_write_kb":2066,"num_scrub_errors":0,"num_shallow_scrub_errors":2,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":321,"ondisk_log_size":321,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":3452,"kb_used_data":3452,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156496516,"statfs":{"total":160255967232,"available":160252432384,"internally_reserved":0,"allocated":3534848,"data_stored":3534848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002139"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:57.893359+0000","last_change":"2026-02-20T11:43:57.907991+0000","last_active":"2026-02-20T11:43:57.893359+0000","last_peered":"2026-02-20T11:43:57.893359+0000","last_clean":"2026-02-20T11:43:57.893359+0000","last_became_active":"2026-02-20T11:42:49.514790+0000","last_became_peered":"2026-02-20T11:42:49.514790+0000","last_unstale":"2026-02-20T11:43:57.893359+0000","last_undegraded":"2026-02-20T11:43:57.893359+0000","last_fullsized":"2026-02-20T11:43:57.893359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:43:57.907933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:43:57.907933+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:57.893195+0000","last_change":"2026-02-20T11:43:57.906444+0000","last_active":"2026-02-20T11:43:57.893195+0000","last_peered":"2026-02-20T11:43:57.893195+0000","last_clean":"2026-02-20T11:43:57.893195+0000","last_became_active":"2026-02-20T11:42:49.514448+0000","last_became_peered":"2026-02-20T11:42:49.514448+0000","last_unstale":"2026-02-20T11:43:57.893195+0000","last_undegraded":"2026-02-20T11:43:57.893195+0000","last_fullsized":"2026-02-20T11:43:57.893195+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2026-02-20T11:43:57.906309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:43:57.906309+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"21'3","reported_seq":43,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:57.893250+0000","last_change":"2026-02-20T11:43:57.905918+0000","last_active":"2026-02-20T11:43:57.893250+0000","last_peered":"2026-02-20T11:43:57.893250+0000","last_clean":"2026-02-20T11:43:57.893250+0000","last_became_active":"2026-02-20T11:42:49.519266+0000","last_became_peered":"2026-02-20T11:42:49.519266+0000","last_unstale":"2026-02-20T11:43:57.893250+0000","last_undegraded":"2026-02-20T11:43:57.893250+0000","last_fullsized":"2026-02-20T11:43:57.893250+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'3","last_scrub_stamp":"2026-02-20T11:43:57.905854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:43:57.905854+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":6,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'2","reported_seq":35,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:57.892423+0000","last_change":"2026-02-20T11:43:57.907397+0000","last_active":"2026-02-20T11:43:57.892423+0000","last_peered":"2026-02-20T11:43:57.892423+0000","last_clean":"2026-02-20T11:43:57.892423+0000","last_became_active":"2026-02-20T11:42:49.518650+0000","last_became_peered":"2026-02-20T11:42:49.518650+0000","last_unstale":"2026-02-20T11:43:57.892423+0000","last_undegraded":"2026-02-20T11:43:57.892423+0000","last_fullsized":"2026-02-20T11:43:57.892423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'2","last_scrub_stamp":"2026-02-20T11:43:57.907337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:43:57.907337+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'18","reported_seq":143,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:57.589669+0000","last_change":"2026-02-20T11:43:57.599827+0000","last_active":"2026-02-20T11:43:57.589669+0000","last_peered":"2026-02-20T11:43:57.589669+0000","last_clean":"2026-02-20T11:43:57.589669+0000","last_became_active":"2026-02-20T11:42:49.529853+0000","last_became_peered":"2026-02-20T11:42:49.529853+0000","last_unstale":"2026-02-20T11:43:57.589669+0000","last_undegraded":"2026-02-20T11:43:57.589669+0000","last_fullsized":"2026-02-20T11:43:57.589669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'18","last_scrub_stamp":"2026-02-20T11:43:57.599739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:43:57.599739+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":236,"num_read_kb":41,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'1","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:58.194544+0000","last_change":"2026-02-20T11:43:58.201748+0000","last_active":"2026-02-20T11:43:58.194544+0000","last_peered":"2026-02-20T11:43:58.194544+0000","last_clean":"2026-02-20T11:43:58.194544+0000","last_became_active":"2026-02-20T11:42:49.531735+0000","last_became_peered":"2026-02-20T11:42:49.531735+0000","last_unstale":"2026-02-20T11:43:58.194544+0000","last_undegraded":"2026-02-20T11:43:58.194544+0000","last_fullsized":"2026-02-20T11:43:58.194544+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2026-02-20T11:43:58.201647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:43:58.201647+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"21'239","reported_seq":505,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:58.193560+0000","last_change":"2026-02-20T11:43:58.201088+0000","last_active":"2026-02-20T11:43:58.193560+0000","last_peered":"2026-02-20T11:43:58.193560+0000","last_clean":"2026-02-20T11:43:58.193560+0000","last_became_active":"2026-02-20T11:42:49.536183+0000","last_became_peered":"2026-02-20T11:42:49.536183+0000","last_unstale":"2026-02-20T11:43:58.193560+0000","last_undegraded":"2026-02-20T11:43:58.193560+0000","last_fullsized":"2026-02-20T11:43:58.193560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'239","last_scrub_stamp":"2026-02-20T11:43:58.201019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:43:58.201019+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":237,"num_read_kb":948,"num_write":239,"num_write_kb":952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"21'5","reported_seq":38,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:57.892245+0000","last_change":"2026-02-20T11:43:57.905535+0000","last_active":"2026-02-20T11:43:57.892245+0000","last_peered":"2026-02-20T11:43:57.892245+0000","last_clean":"2026-02-20T11:43:57.892245+0000","last_became_active":"2026-02-20T11:42:49.511262+0000","last_became_peered":"2026-02-20T11:42:49.511262+0000","last_unstale":"2026-02-20T11:43:57.892245+0000","last_undegraded":"2026-02-20T11:43:57.892245+0000","last_fullsized":"2026-02-20T11:43:57.892245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'5","last_scrub_stamp":"2026-02-20T11:43:57.905448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:43:57.905448+0000","last_clean_scrub_stamp":"2026-02-20T11:42:48.470549+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":104,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-20T11:43:57.891403+0000","last_change":"2026-02-20T11:43:57.924544+0000","last_active":"2026-02-20T11:43:57.891403+0000","last_peered":"2026-02-20T11:43:57.891403+0000","last_clean":"2026-02-20T11:43:57.891403+0000","last_became_active":"2026-02-20T11:42:45.491808+0000","last_became_peered":"2026-02-20T11:42:45.491808+0000","last_unstale":"2026-02-20T11:43:57.891403+0000","last_undegraded":"2026-02-20T11:43:57.891403+0000","last_fullsized":"2026-02-20T11:43:57.891403+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'52","last_scrub_stamp":"2026-02-20T11:43:57.924476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:43:57.924476+0000","last_clean_scrub_stamp":"2026-02-20T11:42:44.456945+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":506,"num_read_kb":995,"num_write":251,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":2,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160252432384,"internally_reserved":0,"allocated":3534848,"data_stored":3534848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417054208,"internally_reserved":0,"allocated":1601536,"data_stored":1601536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":548,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166108,"statfs":{"total":53418655744,"available":53418094592,"internally_reserved":0,"allocated":561152,"data_stored":561152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1564,"kb_used_data":1564,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165092,"statfs":{"total":53418655744,"available":53417054208,"internally_reserved":0,"allocated":1601536,"data_stored":1601536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1340,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165316,"statfs":{"total":53418655744,"available":53417283584,"internally_reserved":0,"allocated":1372160,"data_stored":1372160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417054208,"internally_reserved":0,"allocated":1601536,"data_stored":1601536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417283584,"internally_reserved":0,"allocated":1372160,"data_stored":1372160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417054208,"internally_reserved":0,"allocated":1601536,"data_stored":1601536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418094592,"internally_reserved":0,"allocated":561152,"data_stored":561152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:44:18.621 DEBUG:teuthology.orchestra.run.trial099:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-20T11:44:18.694 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:44:18.688+0000 7fdb436cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:44:18.696 INFO:teuthology.orchestra.run.trial099.stderr:2026-02-20T11:44:18.694+0000 7fdb436cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:44:18.822 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-20T11:44:18.822 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-20T11:44:18.823 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-20T11:44:18.823 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T11:44:18.888 INFO:tasks.ceph.osd.0:Stopped 2026-02-20T11:44:18.889 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-20T11:44:18.889 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T11:44:18.954 INFO:tasks.ceph.osd.1:Stopped 2026-02-20T11:44:18.954 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-20T11:44:18.954 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T11:44:19.016 INFO:tasks.ceph.osd.2:Stopped 2026-02-20T11:44:19.016 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-20T11:44:19.016 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-20T11:44:19.016 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T11:44:19.049 INFO:tasks.ceph.mgr.x:Stopped 2026-02-20T11:44:19.049 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-20T11:44:19.049 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-20T11:44:19.049 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T11:44:19.101 INFO:tasks.ceph.mon.a:Stopped 2026-02-20T11:44:19.101 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-20T11:44:19.101 DEBUG:teuthology.orchestra.run.trial099:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2026-02-20T11:44:19.167 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial099.front.sepia.ceph.com 2026-02-20T11:44:19.168 DEBUG:teuthology.orchestra.run.trial099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-20T11:44:19.288 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial099.front.sepia.ceph.com 2026-02-20T11:44:19.288 DEBUG:teuthology.orchestra.run.trial099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-20T11:44:19.315 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial099.front.sepia.ceph.com 2026-02-20T11:44:19.315 DEBUG:teuthology.orchestra.run.trial099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-20T11:44:19.385 INFO:tasks.ceph:Archiving mon data... 2026-02-20T11:44:19.387 DEBUG:teuthology.misc:Transferring archived files from trial099:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60546/data/mon.a.tgz 2026-02-20T11:44:19.387 DEBUG:teuthology.orchestra.run.trial099:> mktemp 2026-02-20T11:44:19.441 INFO:teuthology.orchestra.run.trial099.stdout:/tmp/tmp.hKftFZCpIm 2026-02-20T11:44:19.441 DEBUG:teuthology.orchestra.run.trial099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.hKftFZCpIm 2026-02-20T11:44:19.527 DEBUG:teuthology.orchestra.run.trial099:> sudo chmod 0666 /tmp/tmp.hKftFZCpIm 2026-02-20T11:44:19.648 DEBUG:teuthology.orchestra.remote:trial099:/tmp/tmp.hKftFZCpIm is 177KB 2026-02-20T11:44:19.701 DEBUG:teuthology.orchestra.run.trial099:> rm -fr /tmp/tmp.hKftFZCpIm 2026-02-20T11:44:19.716 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-20T11:44:19.717 DEBUG:teuthology.orchestra.run.trial099:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-20T11:44:19.837 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-20T11:44:19.837 INFO:tasks.ceph:Archiving crash dumps... 2026-02-20T11:44:19.840 DEBUG:teuthology.misc:Transferring archived files from trial099:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60546/remote/trial099/crash 2026-02-20T11:44:19.842 DEBUG:teuthology.orchestra.run.trial099:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-20T11:44:19.901 INFO:tasks.ceph:Compressing logs... 2026-02-20T11:44:19.901 DEBUG:teuthology.orchestra.run.trial099:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-20T11:44:19.983 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42153.log 2026-02-20T11:44:19.983 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-20T11:44:19.983 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42153.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-20T11:44:19.983 INFO:teuthology.orchestra.run.trial099.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gz 2026-02-20T11:44:19.984 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41522.log 2026-02-20T11:44:19.984 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42573.log 2026-02-20T11:44:19.984 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5/var/log/ceph/ceph-client.admin.41522.log: --verbose -- /var/log/ceph/ceph-client.admin.42671.log 2026-02-20T11:44:19.984 INFO:teuthology.orchestra.run.trial099.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41522.log.gz 2026-02-20T11:44:19.984 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42573.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-20T11:44:19.984 INFO:teuthology.orchestra.run.trial099.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42573.log.gz 2026-02-20T11:44:19.984 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42671.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42671.log.gz 2026-02-20T11:44:19.984 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41227.log 2026-02-20T11:44:19.984 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42285.log 2026-02-20T11:44:19.984 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.41227.log: gzip 50.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.41227.log.gz --verbose 2026-02-20T11:44:19.984 INFO:teuthology.orchestra.run.trial099.stderr: -- /var/log/ceph/ceph-client.admin.43014.log 2026-02-20T11:44:19.985 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42238.log 2026-02-20T11:44:19.985 INFO:teuthology.orchestra.run.trial099.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.42285.log.gz 2026-02-20T11:44:19.985 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.43014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43061.log 2026-02-20T11:44:19.985 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43014.log.gz 2026-02-20T11:44:19.985 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42238.log: /var/log/ceph/ceph-client.admin.42769.log 2026-02-20T11:44:19.985 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42215.log 2026-02-20T11:44:19.985 INFO:teuthology.orchestra.run.trial099.stderr: 66.7%/var/log/ceph/ceph-client.admin.43061.log: -- replaced with /var/log/ceph/ceph-client.admin.42238.log.gz 2026-02-20T11:44:19.985 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42769.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43061.log.gz 2026-02-20T11:44:19.985 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42769.log.gz 2026-02-20T11:44:19.985 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40818.log 2026-02-20T11:44:19.985 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41125.log 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42215.log: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.42215.log.gz 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.40818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42965.log 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40818.log.gz 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40108.log 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41125.log.gz 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42965.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42965.log.gz 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42622.log 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42475.log 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.40108.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40108.log.gz 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42260.log 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42622.log.gz 2026-02-20T11:44:19.986 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40061.log 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42475.log.gz 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42152.log 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42260.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42260.log.gz 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.40061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41076.log 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr: 46.8% -- replaced with /var/log/ceph/ceph-client.admin.40061.log.gz 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41615.log 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr: 65.1% -- replaced with /var/log/ceph/ceph-client.admin.42152.log.gz 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41027.log 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41076.log.gz 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40589.log 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41615.log.gz 2026-02-20T11:44:19.987 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40558.log 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41027.log.gz 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5/var/log/ceph/ceph-client.admin.40589.log: --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40589.log.gz 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40245.log 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.40558.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40558.log.gz 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42193.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gz 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42916.log 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.40245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41669.log 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40245.log.gz 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42916.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42916.log.gz 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42720.log 2026-02-20T11:44:19.988 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42867.log 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41669.log.gz 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41668.log 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42720.log.gz 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42867.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42867.log.gz 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40978.log 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41668.log.gz 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40292.log/var/log/ceph/ceph.audit.log: 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.40978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40913.log 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40978.log.gz 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr: 86.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.40292.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40292.log.gz 2026-02-20T11:44:19.989 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42330.log 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.40913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40743.log 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.40913.log.gz 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42330.log: gzip 66.7% -- replaced with /var/log/ceph/ceph-client.admin.42330.log.gz -5 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41762.log 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.40743.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40743.log.gz 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39966.log 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41762.log.gz 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41493.log 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.39966.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39966.log.gz 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41811.log 2026-02-20T11:44:19.990 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41493.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41493.log.gz 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42524.log 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41373.log 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41811.log.gz 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42309.log 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42524.log.gz 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41375.log 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41373.log.gz 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42309.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.42309.log.gz 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40888.log 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41228.log 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41375.log.gz 2026-02-20T11:44:19.991 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.40888.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40888.log.gz 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41226.log 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41228.log.gz 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41374.log 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42135.log 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41226.log.gz 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41374.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41374.log.gz 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38672.log 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42135.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr: 66.4% -- replaced with /var/log/ceph/ceph-client.admin.42135.log.gz 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph.tmp-client.admin.38672.log: gzip 68.0% -5 -- replaced with /var/log/ceph/ceph.tmp-client.admin.38672.log.gz --verbose 2026-02-20T11:44:19.992 INFO:teuthology.orchestra.run.trial099.stderr: -- /var/log/ceph/ceph-client.admin.42818.log 2026-02-20T11:44:19.993 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41860.log 2026-02-20T11:44:19.993 INFO:teuthology.orchestra.run.trial099.stderr: 84.3% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-20T11:44:19.993 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.42818.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42818.log.gz 2026-02-20T11:44:19.993 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40339.log 2026-02-20T11:44:19.993 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41495.log 2026-02-20T11:44:19.993 INFO:teuthology.orchestra.run.trial099.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41860.log.gz 2026-02-20T11:44:19.993 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.40339.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.40339.log.gz 2026-02-20T11:44:19.993 INFO:teuthology.orchestra.run.trial099.stderr:/var/log/ceph/ceph-client.admin.41495.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41495.log.gz 2026-02-20T11:44:19.994 INFO:teuthology.orchestra.run.trial099.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-20T11:44:19.995 INFO:teuthology.orchestra.run.trial099.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-20T11:44:20.261 INFO:teuthology.orchestra.run.trial099.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-20T11:44:20.335 INFO:teuthology.orchestra.run.trial099.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-20T11:44:20.336 INFO:teuthology.orchestra.run.trial099.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-20T11:44:20.337 INFO:teuthology.orchestra.run.trial099.stderr: 2026-02-20T11:44:20.338 INFO:teuthology.orchestra.run.trial099.stderr:real 0m0.381s 2026-02-20T11:44:20.338 INFO:teuthology.orchestra.run.trial099.stderr:user 0m1.014s 2026-02-20T11:44:20.338 INFO:teuthology.orchestra.run.trial099.stderr:sys 0m0.104s 2026-02-20T11:44:20.339 INFO:tasks.ceph:Archiving logs... 2026-02-20T11:44:20.339 DEBUG:teuthology.misc:Transferring archived files from trial099:/var/log/ceph to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60546/remote/trial099/log 2026-02-20T11:44:20.341 DEBUG:teuthology.orchestra.run.trial099:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-20T11:44:20.688 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-20T11:44:20.863 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-20T11:44:20.875 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-20T11:44:20.875 DEBUG:teuthology.orchestra.run.trial099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-20T11:44:20.901 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-20T11:44:20.912 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-20T11:44:20.912 DEBUG:teuthology.orchestra.run.trial099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T11:44:20.956 INFO:teuthology.orchestra.run.trial099.stderr:bash: line 1: ntpq: command not found 2026-02-20T11:44:21.256 INFO:teuthology.orchestra.run.trial099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T11:44:21.256 INFO:teuthology.orchestra.run.trial099.stdout:=============================================================================== 2026-02-20T11:44:21.256 INFO:teuthology.orchestra.run.trial099.stdout:^+ clock.xmission.com 2 6 377 8 -97us[ -97us] +/- 65ms 2026-02-20T11:44:21.257 INFO:teuthology.orchestra.run.trial099.stdout:^+ edge-anycast.ntp00.hosti> 2 6 377 9 -479us[ -46us] +/- 38ms 2026-02-20T11:44:21.257 INFO:teuthology.orchestra.run.trial099.stdout:^+ time.cloudflare.com 3 6 377 9 -4033us[-3600us] +/- 58ms 2026-02-20T11:44:21.257 INFO:teuthology.orchestra.run.trial099.stdout:^* 149.28.61.105.vultruserc> 3 6 377 9 +608us[+1041us] +/- 32ms 2026-02-20T11:44:21.258 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-20T11:44:21.274 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-20T11:44:21.274 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-20T11:44:21.289 DEBUG:teuthology.orchestra.run.trial099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-20T11:44:21.354 DEBUG:teuthology.orchestra.run.trial099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T11:44:21.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:44:21.381 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-20T11:44:21.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-20T11:44:21.410 INFO:teuthology.task.internal:Duration was 764.868587 seconds 2026-02-20T11:44:21.411 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-20T11:44:21.427 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-20T11:44:21.427 DEBUG:teuthology.orchestra.run.trial099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-20T11:44:21.464 INFO:teuthology.orchestra.run.trial099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T11:44:21.780 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-20T11:44:21.780 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial099.front.sepia.ceph.com 2026-02-20T11:44:21.780 DEBUG:teuthology.orchestra.run.trial099:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-20T11:44:21.840 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-20T11:44:21.840 DEBUG:teuthology.orchestra.run.trial099:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-20T11:44:21.943 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-20T11:44:21.943 DEBUG:teuthology.orchestra.run.trial099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-20T11:44:21.965 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T11:44:21.966 INFO:teuthology.orchestra.run.trial099.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T11:44:21.966 INFO:teuthology.orchestra.run.trial099.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-20T11:44:21.966 INFO:teuthology.orchestra.run.trial099.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-20T11:44:21.966 INFO:teuthology.orchestra.run.trial099.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-20T11:44:21.977 INFO:teuthology.orchestra.run.trial099.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-20T11:44:21.979 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-20T11:44:21.991 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-20T11:44:21.991 DEBUG:teuthology.orchestra.run.trial099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-20T11:44:22.044 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-20T11:44:22.061 DEBUG:teuthology.orchestra.run.trial099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-20T11:44:22.108 INFO:teuthology.orchestra.run.trial099.stdout:kernel.core_pattern = core 2026-02-20T11:44:22.123 DEBUG:teuthology.orchestra.run.trial099:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-20T11:44:22.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:44:22.180 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-20T11:44:22.198 INFO:teuthology.task.internal:Transferring archived files... 2026-02-20T11:44:22.199 DEBUG:teuthology.misc:Transferring archived files from trial099:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60546/remote/trial099 2026-02-20T11:44:22.200 DEBUG:teuthology.orchestra.run.trial099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-20T11:44:22.254 INFO:teuthology.task.internal:Removing archive directory... 2026-02-20T11:44:22.254 DEBUG:teuthology.orchestra.run.trial099:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-20T11:44:22.303 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-20T11:44:22.315 INFO:teuthology.task.internal:Not uploading archives. 2026-02-20T11:44:22.316 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-20T11:44:22.325 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-20T11:44:22.326 DEBUG:teuthology.orchestra.run.trial099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-20T11:44:22.359 INFO:teuthology.orchestra.run.trial099.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 20 11:44 /home/ubuntu/cephtest 2026-02-20T11:44:22.360 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-20T11:44:22.372 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-20T11:44:22.396 INFO:teuthology.run:Summary data: description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rbd_lock_and_fence} duration: 764.8685867786407 flavor: debug owner: scheduled_jjperez@soko04.front.sepia.ceph.com success: true 2026-02-20T11:44:22.396 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T11:44:22.566 INFO:teuthology.run:pass