2026-02-20T11:27:25.409 INFO:root:teuthology version: 1.2.4.dev11+ge6d9099ce 2026-02-20T11:27:25.423 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T11:27:25.502 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60490 branch: wip-perezjos-crimson-only-19-02-2026-PR64439 description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rbd_cls_tests} email: null first_in_suite: false flavor: debug job_id: '60490' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-perezjos-crimson-only-19-02-2026-PR64439 ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 445d5c9ba18a4040e70e25303230c36c77773acb owner: scheduled_jjperez@soko04.front.sepia.ceph.com priority: 101 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 1736 sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 445d5c9ba18a4040e70e25303230c36c77773acb targets: trial127.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCfbDtBOKvQP5IdSL2H5iB3GhHqW3owcH1ZPa8tpvf4XdZLs2vELWgT9Hs0209l3MQdgxsrX+tMIH5OMUALxp18= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 - workunit: clients: client.0: - cls/test_cls_rbd.sh - cls/test_cls_lock.sh - cls/test_cls_journal.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: reimage-ssh-down teuthology_sha1: e6d9099ce03dc21193fda653159ec82465c3a94a timestamp: 2026-02-20_10:21:31 tube: trial user: jjperez verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-20T11:27:25.503 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa; will attempt to use it 2026-02-20T11:27:25.503 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks 2026-02-20T11:27:25.503 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-20T11:27:25.505 INFO:teuthology.task.internal:Checking packages... 2026-02-20T11:27:25.505 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4' 2026-02-20T11:27:25.505 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-20T11:27:25.505 INFO:teuthology.packaging:ref: None 2026-02-20T11:27:25.505 INFO:teuthology.packaging:tag: None 2026-02-20T11:27:25.505 INFO:teuthology.packaging:branch: wip-perezjos-crimson-only-19-02-2026-PR64439 2026-02-20T11:27:25.505 INFO:teuthology.packaging:sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:27:25.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-perezjos-crimson-only-19-02-2026-PR64439 2026-02-20T11:27:25.705 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5357.g0c3148ca 2026-02-20T11:27:25.707 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-20T11:27:25.718 INFO:teuthology.task.internal:no buildpackages task found 2026-02-20T11:27:25.718 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-20T11:27:25.731 INFO:teuthology.task.internal:Saving configuration 2026-02-20T11:27:25.743 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-20T11:27:25.756 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-20T11:27:25.833 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial127.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60490', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-20 11:10:10.975844', 'locked_by': 'scheduled_jjperez@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCfbDtBOKvQP5IdSL2H5iB3GhHqW3owcH1ZPa8tpvf4XdZLs2vELWgT9Hs0209l3MQdgxsrX+tMIH5OMUALxp18='} 2026-02-20T11:27:25.833 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-20T11:27:25.843 INFO:teuthology.task.internal:roles: ubuntu@trial127.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-20T11:27:25.843 INFO:teuthology.run_tasks:Running task console_log... 2026-02-20T11:27:25.930 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7b6acb123ec0>, signals=[15]) 2026-02-20T11:27:25.930 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-20T11:27:25.941 INFO:teuthology.task.internal:Opening connections... 2026-02-20T11:27:25.941 DEBUG:teuthology.task.internal:connecting to ubuntu@trial127.front.sepia.ceph.com 2026-02-20T11:27:25.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial127.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T11:27:26.003 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-20T11:27:26.014 DEBUG:teuthology.orchestra.run.trial127:> uname -m 2026-02-20T11:27:26.076 INFO:teuthology.orchestra.run.trial127.stdout:x86_64 2026-02-20T11:27:26.076 DEBUG:teuthology.orchestra.run.trial127:> cat /etc/os-release 2026-02-20T11:27:26.133 INFO:teuthology.orchestra.run.trial127.stdout:NAME="CentOS Stream" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:VERSION="9" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:ID="centos" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:ID_LIKE="rhel fedora" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:VERSION_ID="9" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:PLATFORM_ID="platform:el9" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:ANSI_COLOR="0;31" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:LOGO="fedora-logo-icon" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:HOME_URL="https://centos.org/" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-20T11:27:26.134 INFO:teuthology.orchestra.run.trial127.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-20T11:27:26.135 INFO:teuthology.lock.ops:Updating trial127.front.sepia.ceph.com on lock server 2026-02-20T11:27:26.228 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-20T11:27:26.239 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-20T11:27:26.251 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-20T11:27:26.251 DEBUG:teuthology.orchestra.run.trial127:> test '!' -e /home/ubuntu/cephtest 2026-02-20T11:27:26.268 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-20T11:27:26.280 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-20T11:27:26.281 DEBUG:teuthology.orchestra.run.trial127:> test -z $(ls -A /var/lib/ceph) 2026-02-20T11:27:26.326 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-20T11:27:26.430 INFO:teuthology.run_tasks:Running task kernel... 2026-02-20T11:27:26.454 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-20T11:27:26.454 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-20T11:27:26.455 DEBUG:teuthology.orchestra.run.trial127:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-20T11:27:26.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:27:26.470 DEBUG:teuthology.orchestra.run.trial127:> uname -r 2026-02-20T11:27:26.528 INFO:teuthology.orchestra.run.trial127.stdout:5.14.0-681.el9.x86_64 2026-02-20T11:27:26.529 INFO:teuthology.task.kernel:Running kernel on trial127: 5.14.0-681.el9.x86_64 2026-02-20T11:27:26.529 DEBUG:teuthology.orchestra.run.trial127:> sudo yum install -y kernel 2026-02-20T11:27:27.917 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - BaseOS 8.9 MB/s | 8.9 MB 00:01 2026-02-20T11:27:31.675 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 27 MB 00:03 2026-02-20T11:27:37.102 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 7.8 MB 00:03 2026-02-20T11:27:38.055 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 20 kB 00:00 2026-02-20T11:27:38.981 INFO:teuthology.orchestra.run.trial127.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2026-02-20T11:27:42.252 INFO:teuthology.orchestra.run.trial127.stdout:lab-extras 463 kB/s | 50 kB 00:00 2026-02-20T11:27:43.261 INFO:teuthology.orchestra.run.trial127.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-20T11:27:43.261 INFO:teuthology.orchestra.run.trial127.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-20T11:27:43.280 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:27:43.281 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:27:43.281 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:27:43.304 DEBUG:teuthology.orchestra.run.trial127:> echo no | sudo yum reinstall kernel || true 2026-02-20T11:27:43.608 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 11:27:42 AM UTC. 2026-02-20T11:27:43.688 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:27:43.690 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:27:43.690 INFO:teuthology.orchestra.run.trial127.stdout: Package Architecture Version Repository Size 2026-02-20T11:27:43.690 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:27:43.690 INFO:teuthology.orchestra.run.trial127.stdout:Reinstalling: 2026-02-20T11:27:43.690 INFO:teuthology.orchestra.run.trial127.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T11:27:43.690 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:27:43.690 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:27:43.690 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:27:43.691 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:27:43.691 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 803 k 2026-02-20T11:27:43.691 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 0 2026-02-20T11:27:43.691 INFO:teuthology.orchestra.run.trial127.stderr:Operation aborted. 2026-02-20T11:27:43.711 INFO:teuthology.orchestra.run.trial127.stdout:Is this ok [y/N]: 2026-02-20T11:27:43.711 DEBUG:teuthology.orchestra.run.trial127:> sudo yum reinstall -y kernel || true 2026-02-20T11:27:44.009 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:27:42 AM UTC. 2026-02-20T11:27:44.089 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:27:44.090 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:27:44.090 INFO:teuthology.orchestra.run.trial127.stdout: Package Architecture Version Repository Size 2026-02-20T11:27:44.090 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:27:44.090 INFO:teuthology.orchestra.run.trial127.stdout:Reinstalling: 2026-02-20T11:27:44.090 INFO:teuthology.orchestra.run.trial127.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T11:27:44.090 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:27:44.090 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:27:44.091 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:27:44.091 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:27:44.091 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 803 k 2026-02-20T11:27:44.091 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 0 2026-02-20T11:27:44.091 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:27:45.008 INFO:teuthology.orchestra.run.trial127.stdout:kernel-5.14.0-681.el9.x86_64.rpm 979 kB/s | 803 kB 00:00 2026-02-20T11:27:45.009 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:27:45.009 INFO:teuthology.orchestra.run.trial127.stdout:Total 874 kB/s | 803 kB 00:00 2026-02-20T11:27:45.019 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:27:45.056 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:27:45.056 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:27:45.061 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:27:45.061 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:27:45.085 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:27:45.089 INFO:teuthology.orchestra.run.trial127.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T11:27:45.089 INFO:teuthology.orchestra.run.trial127.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:27:45.197 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:27:45.198 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T11:27:45.254 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:27:45.254 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:27:45.254 INFO:teuthology.orchestra.run.trial127.stdout:Reinstalled: 2026-02-20T11:27:45.254 INFO:teuthology.orchestra.run.trial127.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-20T11:27:45.254 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:27:45.254 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:27:45.281 DEBUG:teuthology.orchestra.run.trial127:> rpm -q kernel | sort -rV | head -n 1 2026-02-20T11:27:45.306 INFO:teuthology.orchestra.run.trial127.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-20T11:27:45.306 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-20T11:27:45.306 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-20T11:27:45.307 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-20T11:27:45.307 DEBUG:teuthology.orchestra.run.trial127:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-20T11:27:45.370 INFO:teuthology.orchestra.run.trial127.stdout:ttyS1 2026-02-20T11:27:45.398 DEBUG:teuthology.parallel:result is None 2026-02-20T11:27:45.398 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-20T11:27:45.408 INFO:teuthology.task.internal:Creating test directory... 2026-02-20T11:27:45.408 DEBUG:teuthology.orchestra.run.trial127:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-20T11:27:45.456 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-20T11:27:45.468 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-20T11:27:45.481 INFO:teuthology.task.internal:Creating archive directory... 2026-02-20T11:27:45.481 DEBUG:teuthology.orchestra.run.trial127:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-20T11:27:45.520 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-20T11:27:45.535 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-20T11:27:45.536 DEBUG:teuthology.orchestra.run.trial127:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-20T11:27:45.581 INFO:teuthology.orchestra.run.trial127.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T11:27:45.589 INFO:teuthology.orchestra.run.trial127.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T11:27:45.591 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-20T11:27:45.604 INFO:teuthology.task.internal:Configuring sudo... 2026-02-20T11:27:45.604 DEBUG:teuthology.orchestra.run.trial127:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-20T11:27:45.656 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-20T11:27:45.672 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-20T11:27:45.673 DEBUG:teuthology.orchestra.run.trial127:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-20T11:27:45.712 DEBUG:teuthology.orchestra.run.trial127:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T11:27:45.874 DEBUG:teuthology.orchestra.run.trial127:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T11:27:45.900 DEBUG:teuthology.orchestra.run.trial127:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T11:27:46.054 DEBUG:teuthology.orchestra.run.trial127:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T11:27:46.079 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:27:46.079 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-20T11:27:46.145 DEBUG:teuthology.orchestra.run.trial127:> sudo service rsyslog restart 2026-02-20T11:27:46.215 INFO:teuthology.orchestra.run.trial127.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T11:27:46.686 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-20T11:27:46.705 INFO:teuthology.task.internal:Starting timer... 2026-02-20T11:27:46.706 INFO:teuthology.run_tasks:Running task pcp... 2026-02-20T11:27:46.727 INFO:teuthology.run_tasks:Running task selinux... 2026-02-20T11:27:46.747 DEBUG:teuthology.orchestra.run.trial127:> sudo service auditd rotate 2026-02-20T11:27:46.774 INFO:teuthology.orchestra.run.trial127.stdout:Rotating logs: 2026-02-20T11:27:46.776 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-20T11:27:46.776 DEBUG:teuthology.orchestra.run.trial127:> /usr/sbin/getenforce 2026-02-20T11:27:46.832 INFO:teuthology.orchestra.run.trial127.stdout:Permissive 2026-02-20T11:27:46.833 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial127.front.sepia.ceph.com': 'permissive'} 2026-02-20T11:27:46.833 DEBUG:teuthology.orchestra.run.trial127:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T11:27:46.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:27:46.898 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-20T11:27:46.898 DEBUG:teuthology.orchestra.run.trial127:> sudo /usr/sbin/setenforce permissive 2026-02-20T11:27:46.962 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-20T11:27:46.978 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-20T11:27:46.981 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-20T11:27:47.186 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-20T11:27:47.193 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-20T11:27:47.193 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial127.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-20T11:30:31.231 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial127.front.sepia.ceph.com')] 2026-02-20T11:30:31.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial127.front.sepia.ceph.com' 2026-02-20T11:30:31.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial127.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T11:30:31.349 DEBUG:teuthology.orchestra.run.trial127:> true 2026-02-20T11:30:31.425 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial127.front.sepia.ceph.com' 2026-02-20T11:30:31.425 INFO:teuthology.run_tasks:Running task clock... 2026-02-20T11:30:31.438 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-20T11:30:31.438 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-20T11:30:31.438 DEBUG:teuthology.orchestra.run.trial127:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T11:30:31.493 INFO:teuthology.orchestra.run.trial127.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-20T11:30:31.505 INFO:teuthology.orchestra.run.trial127.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-20T11:30:31.525 INFO:teuthology.orchestra.run.trial127.stderr:sudo: ntpd: command not found 2026-02-20T11:30:31.535 INFO:teuthology.orchestra.run.trial127.stdout:506 Cannot talk to daemon 2026-02-20T11:30:31.547 INFO:teuthology.orchestra.run.trial127.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-20T11:30:31.559 INFO:teuthology.orchestra.run.trial127.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-20T11:30:31.611 INFO:teuthology.orchestra.run.trial127.stderr:bash: line 1: ntpq: command not found 2026-02-20T11:30:31.612 INFO:teuthology.orchestra.run.trial127.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T11:30:31.612 INFO:teuthology.orchestra.run.trial127.stdout:=============================================================================== 2026-02-20T11:30:31.614 INFO:teuthology.run_tasks:Running task install... 2026-02-20T11:30:31.625 DEBUG:teuthology.task.install:project ceph 2026-02-20T11:30:31.625 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}} 2026-02-20T11:30:31.625 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'} 2026-02-20T11:30:31.625 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T11:30:31.628 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T11:30:31.628 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-20T11:30:31.628 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'tag': None, 'wait_for_package': False} 2026-02-20T11:30:31.628 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:30:31.745 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/ 2026-02-20T11:30:31.745 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5357.g0c3148ca 2026-02-20T11:30:32.078 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T11:30:32.078 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:30:32.079 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T11:30:32.105 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-20T11:30:32.105 DEBUG:teuthology.orchestra.run.trial127:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T11:30:32.169 DEBUG:teuthology.orchestra.run.trial127:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T11:30:32.243 DEBUG:teuthology.orchestra.run.trial127:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T11:30:32.306 INFO:teuthology.orchestra.run.trial127.stdout:check_obsoletes = 1 2026-02-20T11:30:32.307 DEBUG:teuthology.orchestra.run.trial127:> sudo yum clean all 2026-02-20T11:30:32.469 INFO:teuthology.orchestra.run.trial127.stdout:41 files removed 2026-02-20T11:30:32.484 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-radosgw 2026-02-20T11:30:33.286 INFO:teuthology.orchestra.run.trial127.stdout:ceph packages for x86_64 142 kB/s | 94 kB 00:00 2026-02-20T11:30:33.936 INFO:teuthology.orchestra.run.trial127.stdout:ceph noarch packages 39 kB/s | 24 kB 00:00 2026-02-20T11:30:34.496 INFO:teuthology.orchestra.run.trial127.stdout:ceph source packages 3.8 kB/s | 2.1 kB 00:00 2026-02-20T11:30:36.265 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.9 MB 00:01 2026-02-20T11:30:39.769 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 27 MB 00:02 2026-02-20T11:30:43.098 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 7.8 MB 00:00 2026-02-20T11:30:44.021 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - Extras packages 108 kB/s | 20 kB 00:00 2026-02-20T11:30:44.765 INFO:teuthology.orchestra.run.trial127.stdout:Extra Packages for Enterprise Linux 30 MB/s | 20 MB 00:00 2026-02-20T11:30:48.029 INFO:teuthology.orchestra.run.trial127.stdout:lab-extras 489 kB/s | 50 kB 00:00 2026-02-20T11:30:49.224 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:30:49.225 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:30:49.225 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repository Size 2026-02-20T11:30:49.226 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:30:49.226 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-20T11:30:49.226 INFO:teuthology.orchestra.run.trial127.stdout: ceph-radosgw x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 11 M 2026-02-20T11:30:49.226 INFO:teuthology.orchestra.run.trial127.stdout:Upgrading: 2026-02-20T11:30:49.226 INFO:teuthology.orchestra.run.trial127.stdout: librados2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 3.3 M 2026-02-20T11:30:49.226 INFO:teuthology.orchestra.run.trial127.stdout: librbd1 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.7 M 2026-02-20T11:30:49.226 INFO:teuthology.orchestra.run.trial127.stdout:Installing dependencies: 2026-02-20T11:30:49.226 INFO:teuthology.orchestra.run.trial127.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-20T11:30:49.226 INFO:teuthology.orchestra.run.trial127.stdout: ceph-base x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 5.1 M 2026-02-20T11:30:49.226 INFO:teuthology.orchestra.run.trial127.stdout: ceph-common x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 21 M 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: ceph-selinux x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 25 k 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 860 k 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: libradosstriper1 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 263 k 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-20T11:30:49.227 INFO:teuthology.orchestra.run.trial127.stdout: librgw2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 5.6 M 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: python3-ceph-argparse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 45 k 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: python3-ceph-common x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 185 k 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: python3-cephfs x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 204 k 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: python3-rados x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 324 k 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: python3-rbd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 305 k 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: python3-rgw x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 100 k 2026-02-20T11:30:49.228 INFO:teuthology.orchestra.run.trial127.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-20T11:30:49.229 INFO:teuthology.orchestra.run.trial127.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-20T11:30:49.229 INFO:teuthology.orchestra.run.trial127.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-20T11:30:49.229 INFO:teuthology.orchestra.run.trial127.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-20T11:30:49.229 INFO:teuthology.orchestra.run.trial127.stdout:Installing weak dependencies: 2026-02-20T11:30:49.229 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-daemon x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 37 k 2026-02-20T11:30:49.229 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 26 k 2026-02-20T11:30:49.229 INFO:teuthology.orchestra.run.trial127.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-20T11:30:49.229 INFO:teuthology.orchestra.run.trial127.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-20T11:30:49.229 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:30:49.229 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:30:49.230 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:30:49.230 INFO:teuthology.orchestra.run.trial127.stdout:Install 35 Packages 2026-02-20T11:30:49.230 INFO:teuthology.orchestra.run.trial127.stdout:Upgrade 2 Packages 2026-02-20T11:30:49.230 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:30:49.231 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 61 M 2026-02-20T11:30:49.231 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:30:50.983 INFO:teuthology.orchestra.run.trial127.stdout:(1/37): ceph-base-20.3.0-5357.g0c3148ca.el9.x86 4.3 MB/s | 5.1 MB 00:01 2026-02-20T11:30:51.044 INFO:teuthology.orchestra.run.trial127.stdout:(2/37): ceph-selinux-20.3.0-5357.g0c3148ca.el9. 415 kB/s | 25 kB 00:00 2026-02-20T11:30:51.102 INFO:teuthology.orchestra.run.trial127.stdout:(3/37): libcephfs-daemon-20.3.0-5357.g0c3148ca. 640 kB/s | 37 kB 00:00 2026-02-20T11:30:51.160 INFO:teuthology.orchestra.run.trial127.stdout:(4/37): libcephfs-proxy2-20.3.0-5357.g0c3148ca. 440 kB/s | 26 kB 00:00 2026-02-20T11:30:51.296 INFO:teuthology.orchestra.run.trial127.stdout:(5/37): libcephfs2-20.3.0-5357.g0c3148ca.el9.x8 6.2 MB/s | 860 kB 00:00 2026-02-20T11:30:51.361 INFO:teuthology.orchestra.run.trial127.stdout:(6/37): libradosstriper1-20.3.0-5357.g0c3148ca. 4.0 MB/s | 263 kB 00:00 2026-02-20T11:30:51.970 INFO:teuthology.orchestra.run.trial127.stdout:(7/37): librgw2-20.3.0-5357.g0c3148ca.el9.x86_6 9.2 MB/s | 5.6 MB 00:00 2026-02-20T11:30:52.024 INFO:teuthology.orchestra.run.trial127.stdout:(8/37): python3-ceph-argparse-20.3.0-5357.g0c31 826 kB/s | 45 kB 00:00 2026-02-20T11:30:52.082 INFO:teuthology.orchestra.run.trial127.stdout:(9/37): python3-ceph-common-20.3.0-5357.g0c3148 3.1 MB/s | 185 kB 00:00 2026-02-20T11:30:52.141 INFO:teuthology.orchestra.run.trial127.stdout:(10/37): python3-cephfs-20.3.0-5357.g0c3148ca.e 3.4 MB/s | 204 kB 00:00 2026-02-20T11:30:52.202 INFO:teuthology.orchestra.run.trial127.stdout:(11/37): python3-rados-20.3.0-5357.g0c3148ca.el 5.2 MB/s | 324 kB 00:00 2026-02-20T11:30:52.264 INFO:teuthology.orchestra.run.trial127.stdout:(12/37): python3-rbd-20.3.0-5357.g0c3148ca.el9. 4.9 MB/s | 305 kB 00:00 2026-02-20T11:30:52.321 INFO:teuthology.orchestra.run.trial127.stdout:(13/37): python3-rgw-20.3.0-5357.g0c3148ca.el9. 1.7 MB/s | 100 kB 00:00 2026-02-20T11:30:52.429 INFO:teuthology.orchestra.run.trial127.stdout:(14/37): ceph-radosgw-20.3.0-5357.g0c3148ca.el9 4.3 MB/s | 11 MB 00:02 2026-02-20T11:30:52.467 INFO:teuthology.orchestra.run.trial127.stdout:(15/37): mailcap-2.1.49-5.el9.noarch.rpm 228 kB/s | 33 kB 00:00 2026-02-20T11:30:52.506 INFO:teuthology.orchestra.run.trial127.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 2.4 MB/s | 93 kB 00:00 2026-02-20T11:30:52.527 INFO:teuthology.orchestra.run.trial127.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 4.9 MB/s | 104 kB 00:00 2026-02-20T11:30:52.548 INFO:teuthology.orchestra.run.trial127.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 7.6 MB/s | 164 kB 00:00 2026-02-20T11:30:52.570 INFO:teuthology.orchestra.run.trial127.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 7.1 MB/s | 160 kB 00:00 2026-02-20T11:30:52.590 INFO:teuthology.orchestra.run.trial127.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 2.2 MB/s | 45 kB 00:00 2026-02-20T11:30:52.600 INFO:teuthology.orchestra.run.trial127.stdout:(21/37): nvme-cli-2.16-1.el9.x86_64.rpm 6.7 MB/s | 1.2 MB 00:00 2026-02-20T11:30:52.625 INFO:teuthology.orchestra.run.trial127.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 12 MB/s | 292 kB 00:00 2026-02-20T11:30:52.631 INFO:teuthology.orchestra.run.trial127.stdout:(23/37): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 662 kB 00:00 2026-02-20T11:30:52.650 INFO:teuthology.orchestra.run.trial127.stdout:(24/37): qatlib-25.08.0-2.el9.x86_64.rpm 9.6 MB/s | 240 kB 00:00 2026-02-20T11:30:52.650 INFO:teuthology.orchestra.run.trial127.stdout:(25/37): qatlib-service-25.08.0-2.el9.x86_64.rp 1.8 MB/s | 37 kB 00:00 2026-02-20T11:30:52.734 INFO:teuthology.orchestra.run.trial127.stdout:(26/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 787 kB/s | 66 kB 00:00 2026-02-20T11:30:52.888 INFO:teuthology.orchestra.run.trial127.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.3 MB/s | 308 kB 00:00 2026-02-20T11:30:52.912 INFO:teuthology.orchestra.run.trial127.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.0 MB/s | 25 kB 00:00 2026-02-20T11:30:52.936 INFO:teuthology.orchestra.run.trial127.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2026-02-20T11:30:52.962 INFO:teuthology.orchestra.run.trial127.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2026-02-20T11:30:52.989 INFO:teuthology.orchestra.run.trial127.stdout:(31/37): libarrow-9.0.0-15.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2026-02-20T11:30:53.014 INFO:teuthology.orchestra.run.trial127.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 7.4 MB/s | 191 kB 00:00 2026-02-20T11:30:53.028 INFO:teuthology.orchestra.run.trial127.stdout:(33/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2026-02-20T11:30:53.047 INFO:teuthology.orchestra.run.trial127.stdout:(34/37): thrift-0.15.0-4.el9.x86_64.rpm 50 MB/s | 1.6 MB 00:00 2026-02-20T11:30:53.533 INFO:teuthology.orchestra.run.trial127.stdout:(35/37): librados2-20.3.0-5357.g0c3148ca.el9.x8 6.5 MB/s | 3.3 MB 00:00 2026-02-20T11:30:53.822 INFO:teuthology.orchestra.run.trial127.stdout:(36/37): librbd1-20.3.0-5357.g0c3148ca.el9.x86_ 3.5 MB/s | 2.7 MB 00:00 2026-02-20T11:31:00.775 INFO:teuthology.orchestra.run.trial127.stdout:(37/37): ceph-common-20.3.0-5357.g0c3148ca.el9. 1.9 MB/s | 21 MB 00:10 2026-02-20T11:31:00.777 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:31:00.777 INFO:teuthology.orchestra.run.trial127.stdout:Total 5.3 MB/s | 61 MB 00:11 2026-02-20T11:31:00.879 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:31:00.899 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:31:00.899 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:31:01.151 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:31:01.151 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:31:01.452 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:31:01.595 INFO:teuthology.orchestra.run.trial127.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-20T11:31:01.600 INFO:teuthology.orchestra.run.trial127.stdout: Upgrading : librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:31:01.650 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:31:01.651 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:31:01.664 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:31:01.669 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_6 4/39 2026-02-20T11:31:01.670 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:31:01.697 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:31:01.699 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 6/39 2026-02-20T11:31:01.708 INFO:teuthology.orchestra.run.trial127.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-20T11:31:01.711 INFO:teuthology.orchestra.run.trial127.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-20T11:31:01.714 INFO:teuthology.orchestra.run.trial127.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-20T11:31:01.718 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-20T11:31:01.729 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9. 11/39 2026-02-20T11:31:01.742 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/39 2026-02-20T11:31:01.742 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 13/39 2026-02-20T11:31:01.765 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 13/39 2026-02-20T11:31:01.769 INFO:teuthology.orchestra.run.trial127.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-20T11:31:01.801 INFO:teuthology.orchestra.run.trial127.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-20T11:31:01.805 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-20T11:31:01.811 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-20T11:31:01.813 INFO:teuthology.orchestra.run.trial127.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-20T11:31:01.818 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-20T11:31:01.932 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-20T11:31:01.933 INFO:teuthology.orchestra.run.trial127.stdout: Upgrading : librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 21/39 2026-02-20T11:31:01.960 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 21/39 2026-02-20T11:31:01.962 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 22/39 2026-02-20T11:31:01.969 INFO:teuthology.orchestra.run.trial127.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-20T11:31:02.219 INFO:teuthology.orchestra.run.trial127.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-20T11:31:02.220 INFO:teuthology.orchestra.run.trial127.stdout: Installing : librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 25/39 2026-02-20T11:31:02.236 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 25/39 2026-02-20T11:31:02.241 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 26/39 2026-02-20T11:31:02.245 INFO:teuthology.orchestra.run.trial127.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-20T11:31:02.248 INFO:teuthology.orchestra.run.trial127.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-20T11:31:02.248 INFO:teuthology.orchestra.run.trial127.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-20T11:31:02.266 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-20T11:31:02.385 INFO:teuthology.orchestra.run.trial127.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-20T11:31:02.387 INFO:teuthology.orchestra.run.trial127.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-20T11:31:02.674 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-20T11:31:02.675 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-20T11:31:02.675 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:31:02.722 INFO:teuthology.orchestra.run.trial127.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-20T11:31:02.725 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x8 33/39 2026-02-20T11:31:03.670 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:31:03.673 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:31:03.938 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:31:03.941 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 35/39 2026-02-20T11:31:03.981 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 35/39 2026-02-20T11:31:03.982 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-20T11:31:03.982 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-20T11:31:03.982 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:31:03.985 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:31:08.564 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:31:08.564 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /sys 2026-02-20T11:31:08.564 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /proc 2026-02-20T11:31:08.564 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /mnt 2026-02-20T11:31:08.564 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /var/tmp 2026-02-20T11:31:08.565 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /home 2026-02-20T11:31:08.565 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /root 2026-02-20T11:31:08.565 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /tmp 2026-02-20T11:31:08.565 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:31:09.054 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 37/39 2026-02-20T11:31:09.078 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 37/39 2026-02-20T11:31:09.078 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:31:09.078 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-20T11:31:09.078 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T11:31:09.078 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T11:31:09.079 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:31:09.080 INFO:teuthology.orchestra.run.trial127.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-20T11:31:09.093 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-20T11:31:09.093 INFO:teuthology.orchestra.run.trial127.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:31:09.149 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:31:09.150 INFO:teuthology.orchestra.run.trial127.stdout:Creating group 'qat' with GID 992. 2026-02-20T11:31:09.150 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:31:09.458 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/39 2026-02-20T11:31:09.458 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:31:09.458 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:31:09.458 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 4/39 2026-02-20T11:31:09.459 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:31:09.459 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 6/39 2026-02-20T11:31:09.459 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 7/39 2026-02-20T11:31:09.459 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 8/39 2026-02-20T11:31:09.459 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 9/39 2026-02-20T11:31:09.459 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9. 10/39 2026-02-20T11:31:09.459 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x8 11/39 2026-02-20T11:31:09.459 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/39 2026-02-20T11:31:09.459 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 13/39 2026-02-20T11:31:09.459 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 14/39 2026-02-20T11:31:09.459 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 15/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-20T11:31:09.460 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-20T11:31:09.461 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-20T11:31:09.461 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-20T11:31:09.461 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-20T11:31:09.461 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-20T11:31:09.461 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-20T11:31:09.461 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-20T11:31:09.461 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-20T11:31:09.461 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:31:09.461 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-20T11:31:09.461 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 38/39 2026-02-20T11:31:09.518 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:31:09.518 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout:Upgraded: 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: re2-1:20211101-20.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:31:09.519 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:31:09.562 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-test 2026-02-20T11:31:09.870 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:21 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:31:09.956 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:31:09.957 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:31:09.957 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repository Size 2026-02-20T11:31:09.957 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:31:09.957 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-20T11:31:09.957 INFO:teuthology.orchestra.run.trial127.stdout: ceph-test x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 127 M 2026-02-20T11:31:09.957 INFO:teuthology.orchestra.run.trial127.stdout:Installing dependencies: 2026-02-20T11:31:09.957 INFO:teuthology.orchestra.run.trial127.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-20T11:31:09.958 INFO:teuthology.orchestra.run.trial127.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-20T11:31:09.958 INFO:teuthology.orchestra.run.trial127.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-20T11:31:09.958 INFO:teuthology.orchestra.run.trial127.stdout: libcephsqlite x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 153 k 2026-02-20T11:31:09.958 INFO:teuthology.orchestra.run.trial127.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-20T11:31:09.958 INFO:teuthology.orchestra.run.trial127.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-20T11:31:09.958 INFO:teuthology.orchestra.run.trial127.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-20T11:31:09.958 INFO:teuthology.orchestra.run.trial127.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-20T11:31:09.958 INFO:teuthology.orchestra.run.trial127.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-20T11:31:09.958 INFO:teuthology.orchestra.run.trial127.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-20T11:31:09.958 INFO:teuthology.orchestra.run.trial127.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-20T11:31:09.958 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:31:09.959 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:31:09.959 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:31:09.959 INFO:teuthology.orchestra.run.trial127.stdout:Install 12 Packages 2026-02-20T11:31:09.959 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:31:09.959 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 132 M 2026-02-20T11:31:09.959 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 989 M 2026-02-20T11:31:09.959 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:31:10.527 INFO:teuthology.orchestra.run.trial127.stdout:(1/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 26 MB/s | 2.1 MB 00:00 2026-02-20T11:31:10.535 INFO:teuthology.orchestra.run.trial127.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 2.5 MB/s | 23 kB 00:00 2026-02-20T11:31:10.545 INFO:teuthology.orchestra.run.trial127.stdout:(3/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 9.2 MB/s | 94 kB 00:00 2026-02-20T11:31:10.573 INFO:teuthology.orchestra.run.trial127.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 19 MB/s | 525 kB 00:00 2026-02-20T11:31:10.582 INFO:teuthology.orchestra.run.trial127.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 19 MB/s | 176 kB 00:00 2026-02-20T11:31:10.591 INFO:teuthology.orchestra.run.trial127.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 25 MB/s | 233 kB 00:00 2026-02-20T11:31:10.629 INFO:teuthology.orchestra.run.trial127.stdout:(7/12): protobuf-3.14.0-17.el9.x86_64.rpm 27 MB/s | 1.0 MB 00:00 2026-02-20T11:31:10.638 INFO:teuthology.orchestra.run.trial127.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 32 MB/s | 303 kB 00:00 2026-02-20T11:31:10.643 INFO:teuthology.orchestra.run.trial127.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 14 MB/s | 64 kB 00:00 2026-02-20T11:31:10.783 INFO:teuthology.orchestra.run.trial127.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 871 kB/s | 122 kB 00:00 2026-02-20T11:31:10.927 INFO:teuthology.orchestra.run.trial127.stdout:(11/12): libcephsqlite-20.3.0-5357.g0c3148ca.el 318 kB/s | 153 kB 00:00 2026-02-20T11:32:59.742 INFO:teuthology.orchestra.run.trial127.stdout:(12/12): ceph-test-20.3.0-5357.g0c3148ca.el9.x8 1.2 MB/s | 127 MB 01:49 2026-02-20T11:32:59.748 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:32:59.748 INFO:teuthology.orchestra.run.trial127.stdout:Total 1.2 MB/s | 132 MB 01:49 2026-02-20T11:32:59.815 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:32:59.826 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:32:59.826 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:33:00.257 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:33:00.257 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:33:00.713 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:33:00.722 INFO:teuthology.orchestra.run.trial127.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-20T11:33:00.731 INFO:teuthology.orchestra.run.trial127.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-20T11:33:00.736 INFO:teuthology.orchestra.run.trial127.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-20T11:33:00.740 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-20T11:33:00.742 INFO:teuthology.orchestra.run.trial127.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-20T11:33:00.746 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-20T11:33:00.747 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T11:33:00.763 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T11:33:00.765 INFO:teuthology.orchestra.run.trial127.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-20T11:33:00.774 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-20T11:33:00.783 INFO:teuthology.orchestra.run.trial127.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-20T11:33:00.784 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 11/12 2026-02-20T11:33:07.973 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 11/12 2026-02-20T11:33:07.992 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/12 2026-02-20T11:33:08.796 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/12 2026-02-20T11:33:08.796 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/12 2026-02-20T11:33:08.796 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/12 2026-02-20T11:33:08.796 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-20T11:33:08.797 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-20T11:33:08.797 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-20T11:33:08.797 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-20T11:33:08.797 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-20T11:33:08.797 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-20T11:33:08.797 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-20T11:33:08.797 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-20T11:33:08.797 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-20T11:33:08.851 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-20T11:33:08.851 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:08.851 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-20T11:33:08.851 INFO:teuthology.orchestra.run.trial127.stdout: ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:08.851 INFO:teuthology.orchestra.run.trial127.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-20T11:33:08.851 INFO:teuthology.orchestra.run.trial127.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-20T11:33:08.851 INFO:teuthology.orchestra.run.trial127.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-20T11:33:08.851 INFO:teuthology.orchestra.run.trial127.stdout: libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:08.851 INFO:teuthology.orchestra.run.trial127.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-20T11:33:08.852 INFO:teuthology.orchestra.run.trial127.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-20T11:33:08.852 INFO:teuthology.orchestra.run.trial127.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-20T11:33:08.852 INFO:teuthology.orchestra.run.trial127.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-20T11:33:08.852 INFO:teuthology.orchestra.run.trial127.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-20T11:33:08.852 INFO:teuthology.orchestra.run.trial127.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-20T11:33:08.852 INFO:teuthology.orchestra.run.trial127.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-20T11:33:08.852 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:08.852 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:08.890 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph 2026-02-20T11:33:09.195 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:02:21 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:09.282 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:09.285 INFO:teuthology.orchestra.run.trial127.stdout:======================================================================================= 2026-02-20T11:33:09.285 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repository Size 2026-02-20T11:33:09.285 INFO:teuthology.orchestra.run.trial127.stdout:======================================================================================= 2026-02-20T11:33:09.286 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-20T11:33:09.286 INFO:teuthology.orchestra.run.trial127.stdout: ceph x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 6.5 k 2026-02-20T11:33:09.286 INFO:teuthology.orchestra.run.trial127.stdout:Installing dependencies: 2026-02-20T11:33:09.286 INFO:teuthology.orchestra.run.trial127.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-20T11:33:09.286 INFO:teuthology.orchestra.run.trial127.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 57 k 2026-02-20T11:33:09.286 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mds x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.0 M 2026-02-20T11:33:09.286 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 937 k 2026-02-20T11:33:09.286 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 297 k 2026-02-20T11:33:09.286 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mon x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 4.9 M 2026-02-20T11:33:09.286 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 163 k 2026-02-20T11:33:09.286 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-classic x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 15 M 2026-02-20T11:33:09.287 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-crimson x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 88 M 2026-02-20T11:33:09.287 INFO:teuthology.orchestra.run.trial127.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 17 k 2026-02-20T11:33:09.287 INFO:teuthology.orchestra.run.trial127.stdout: cephadm noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 1.0 M 2026-02-20T11:33:09.287 INFO:teuthology.orchestra.run.trial127.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-20T11:33:09.287 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-20T11:33:09.287 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-20T11:33:09.287 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-20T11:33:09.287 INFO:teuthology.orchestra.run.trial127.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-20T11:33:09.287 INFO:teuthology.orchestra.run.trial127.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-20T11:33:09.287 INFO:teuthology.orchestra.run.trial127.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-20T11:33:09.288 INFO:teuthology.orchestra.run.trial127.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-20T11:33:09.288 INFO:teuthology.orchestra.run.trial127.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-20T11:33:09.288 INFO:teuthology.orchestra.run.trial127.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-20T11:33:09.288 INFO:teuthology.orchestra.run.trial127.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-20T11:33:09.288 INFO:teuthology.orchestra.run.trial127.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-20T11:33:09.288 INFO:teuthology.orchestra.run.trial127.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-20T11:33:09.288 INFO:teuthology.orchestra.run.trial127.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-20T11:33:09.288 INFO:teuthology.orchestra.run.trial127.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-20T11:33:09.288 INFO:teuthology.orchestra.run.trial127.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-20T11:33:09.288 INFO:teuthology.orchestra.run.trial127.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-20T11:33:09.289 INFO:teuthology.orchestra.run.trial127.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-20T11:33:09.290 INFO:teuthology.orchestra.run.trial127.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-20T11:33:09.291 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-20T11:33:09.292 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-20T11:33:09.292 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-20T11:33:09.292 INFO:teuthology.orchestra.run.trial127.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-20T11:33:09.292 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-20T11:33:09.292 INFO:teuthology.orchestra.run.trial127.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-20T11:33:09.292 INFO:teuthology.orchestra.run.trial127.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-20T11:33:09.292 INFO:teuthology.orchestra.run.trial127.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-20T11:33:09.292 INFO:teuthology.orchestra.run.trial127.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-20T11:33:09.293 INFO:teuthology.orchestra.run.trial127.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-20T11:33:09.294 INFO:teuthology.orchestra.run.trial127.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-20T11:33:09.294 INFO:teuthology.orchestra.run.trial127.stdout:Installing weak dependencies: 2026-02-20T11:33:09.294 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 184 k 2026-02-20T11:33:09.294 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 51 M 2026-02-20T11:33:09.294 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 7.4 M 2026-02-20T11:33:09.294 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 22 k 2026-02-20T11:33:09.294 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-rook noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 50 k 2026-02-20T11:33:09.294 INFO:teuthology.orchestra.run.trial127.stdout: ceph-volume noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 301 k 2026-02-20T11:33:09.294 INFO:teuthology.orchestra.run.trial127.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-20T11:33:09.294 INFO:teuthology.orchestra.run.trial127.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-20T11:33:09.294 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:09.295 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:33:09.295 INFO:teuthology.orchestra.run.trial127.stdout:======================================================================================= 2026-02-20T11:33:09.295 INFO:teuthology.orchestra.run.trial127.stdout:Install 96 Packages 2026-02-20T11:33:09.295 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:09.296 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 226 M 2026-02-20T11:33:09.296 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 1.5 G 2026-02-20T11:33:09.296 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:33:10.159 INFO:teuthology.orchestra.run.trial127.stdout:(1/96): ceph-20.3.0-5357.g0c3148ca.el9.x86_64.r 24 kB/s | 6.5 kB 00:00 2026-02-20T11:33:11.042 INFO:teuthology.orchestra.run.trial127.stdout:(2/96): ceph-mon-20.3.0-5357.g0c3148ca.el9.x86_ 5.5 MB/s | 4.9 MB 00:00 2026-02-20T11:33:11.044 INFO:teuthology.orchestra.run.trial127.stdout:(3/96): ceph-mgr-20.3.0-5357.g0c3148ca.el9.x86_ 810 kB/s | 937 kB 00:01 2026-02-20T11:33:11.096 INFO:teuthology.orchestra.run.trial127.stdout:(4/96): ceph-osd-20.3.0-5357.g0c3148ca.el9.x86_ 3.0 MB/s | 163 kB 00:00 2026-02-20T11:33:11.277 INFO:teuthology.orchestra.run.trial127.stdout:(5/96): ceph-mds-20.3.0-5357.g0c3148ca.el9.x86_ 1.5 MB/s | 2.0 MB 00:01 2026-02-20T11:33:11.342 INFO:teuthology.orchestra.run.trial127.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5357.g0c 875 kB/s | 57 kB 00:00 2026-02-20T11:33:11.472 INFO:teuthology.orchestra.run.trial127.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5357.g0c3148ca. 1.4 MB/s | 184 kB 00:00 2026-02-20T11:33:19.229 INFO:teuthology.orchestra.run.trial127.stdout:(8/96): ceph-osd-crimson-20.3.0-5357.g0c3148ca. 11 MB/s | 88 MB 00:08 2026-02-20T11:33:20.439 INFO:teuthology.orchestra.run.trial127.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-53 6.1 MB/s | 7.4 MB 00:01 2026-02-20T11:33:20.504 INFO:teuthology.orchestra.run.trial127.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5357.g0c3148 345 kB/s | 22 kB 00:00 2026-02-20T11:33:20.571 INFO:teuthology.orchestra.run.trial127.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5357.g0c3 4.3 MB/s | 297 kB 00:00 2026-02-20T11:33:20.633 INFO:teuthology.orchestra.run.trial127.stdout:(12/96): ceph-mgr-rook-20.3.0-5357.g0c3148ca.el 812 kB/s | 50 kB 00:00 2026-02-20T11:33:20.694 INFO:teuthology.orchestra.run.trial127.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5357.g0c 283 kB/s | 17 kB 00:00 2026-02-20T11:33:20.760 INFO:teuthology.orchestra.run.trial127.stdout:(14/96): ceph-volume-20.3.0-5357.g0c3148ca.el9. 4.5 MB/s | 301 kB 00:00 2026-02-20T11:33:20.960 INFO:teuthology.orchestra.run.trial127.stdout:(15/96): cephadm-20.3.0-5357.g0c3148ca.el9.noar 5.1 MB/s | 1.0 MB 00:00 2026-02-20T11:33:21.089 INFO:teuthology.orchestra.run.trial127.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 2.7 MB/s | 351 kB 00:00 2026-02-20T11:33:21.111 INFO:teuthology.orchestra.run.trial127.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.8 MB/s | 40 kB 00:00 2026-02-20T11:33:21.134 INFO:teuthology.orchestra.run.trial127.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 3.1 MB/s | 72 kB 00:00 2026-02-20T11:33:21.180 INFO:teuthology.orchestra.run.trial127.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 17 MB/s | 794 kB 00:00 2026-02-20T11:33:21.206 INFO:teuthology.orchestra.run.trial127.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 7.2 MB/s | 184 kB 00:00 2026-02-20T11:33:21.231 INFO:teuthology.orchestra.run.trial127.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 9.9 MB/s | 253 kB 00:00 2026-02-20T11:33:21.295 INFO:teuthology.orchestra.run.trial127.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 19 MB/s | 1.2 MB 00:00 2026-02-20T11:33:21.320 INFO:teuthology.orchestra.run.trial127.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 4.3 MB/s | 106 kB 00:00 2026-02-20T11:33:21.344 INFO:teuthology.orchestra.run.trial127.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 5.4 MB/s | 135 kB 00:00 2026-02-20T11:33:21.369 INFO:teuthology.orchestra.run.trial127.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 6.0 MB/s | 150 kB 00:00 2026-02-20T11:33:21.394 INFO:teuthology.orchestra.run.trial127.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 4.9 MB/s | 126 kB 00:00 2026-02-20T11:33:21.419 INFO:teuthology.orchestra.run.trial127.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 8.5 MB/s | 218 kB 00:00 2026-02-20T11:33:21.444 INFO:teuthology.orchestra.run.trial127.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 7.0 MB/s | 182 kB 00:00 2026-02-20T11:33:21.469 INFO:teuthology.orchestra.run.trial127.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 11 MB/s | 266 kB 00:00 2026-02-20T11:33:21.497 INFO:teuthology.orchestra.run.trial127.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 1.0 MB/s | 30 kB 00:00 2026-02-20T11:33:21.564 INFO:teuthology.orchestra.run.trial127.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 45 MB/s | 3.0 MB 00:00 2026-02-20T11:33:21.573 INFO:teuthology.orchestra.run.trial127.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 1.6 MB/s | 15 kB 00:00 2026-02-20T11:33:21.584 INFO:teuthology.orchestra.run.trial127.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 22 MB/s | 246 kB 00:00 2026-02-20T11:33:21.595 INFO:teuthology.orchestra.run.trial127.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 17 MB/s | 188 kB 00:00 2026-02-20T11:33:21.604 INFO:teuthology.orchestra.run.trial127.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 4.4 MB/s | 42 kB 00:00 2026-02-20T11:33:21.633 INFO:teuthology.orchestra.run.trial127.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 189 MB/s | 5.3 MB 00:00 2026-02-20T11:33:21.667 INFO:teuthology.orchestra.run.trial127.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 175 MB/s | 6.0 MB 00:00 2026-02-20T11:33:21.679 INFO:teuthology.orchestra.run.trial127.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 20 MB/s | 244 kB 00:00 2026-02-20T11:33:21.691 INFO:teuthology.orchestra.run.trial127.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 21 MB/s | 249 kB 00:00 2026-02-20T11:33:21.700 INFO:teuthology.orchestra.run.trial127.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 4.8 MB/s | 48 kB 00:00 2026-02-20T11:33:21.711 INFO:teuthology.orchestra.run.trial127.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 16 MB/s | 177 kB 00:00 2026-02-20T11:33:21.726 INFO:teuthology.orchestra.run.trial127.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 85 MB/s | 1.2 MB 00:00 2026-02-20T11:33:21.735 INFO:teuthology.orchestra.run.trial127.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 3.5 MB/s | 35 kB 00:00 2026-02-20T11:33:21.768 INFO:teuthology.orchestra.run.trial127.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 188 MB/s | 6.1 MB 00:00 2026-02-20T11:33:21.781 INFO:teuthology.orchestra.run.trial127.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 35 MB/s | 442 kB 00:00 2026-02-20T11:33:21.791 INFO:teuthology.orchestra.run.trial127.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 7.4 MB/s | 77 kB 00:00 2026-02-20T11:33:21.802 INFO:teuthology.orchestra.run.trial127.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 25 MB/s | 267 kB 00:00 2026-02-20T11:33:21.813 INFO:teuthology.orchestra.run.trial127.stdout:(48/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 14 MB/s | 159 kB 00:00 2026-02-20T11:33:21.825 INFO:teuthology.orchestra.run.trial127.stdout:(49/96): python3-pyasn1-modules-0.4.8-6.el9.noa 23 MB/s | 279 kB 00:00 2026-02-20T11:33:21.835 INFO:teuthology.orchestra.run.trial127.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 5.3 MB/s | 54 kB 00:00 2026-02-20T11:33:21.922 INFO:teuthology.orchestra.run.trial127.stdout:(51/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 222 MB/s | 19 MB 00:00 2026-02-20T11:33:21.932 INFO:teuthology.orchestra.run.trial127.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 4.0 MB/s | 42 kB 00:00 2026-02-20T11:33:21.942 INFO:teuthology.orchestra.run.trial127.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 18 MB/s | 189 kB 00:00 2026-02-20T11:33:21.952 INFO:teuthology.orchestra.run.trial127.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 8.9 MB/s | 90 kB 00:00 2026-02-20T11:33:22.495 INFO:teuthology.orchestra.run.trial127.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 41 kB/s | 22 kB 00:00 2026-02-20T11:33:23.239 INFO:teuthology.orchestra.run.trial127.stdout:(56/96): ceph-osd-classic-20.3.0-5357.g0c3148ca 1.2 MB/s | 15 MB 00:12 2026-02-20T11:33:23.394 INFO:teuthology.orchestra.run.trial127.stdout:(57/96): protobuf-compiler-3.14.0-17.el9.x86_64 958 kB/s | 862 kB 00:00 2026-02-20T11:33:23.695 INFO:teuthology.orchestra.run.trial127.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2026-02-20T11:33:23.952 INFO:teuthology.orchestra.run.trial127.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 590 kB/s | 151 kB 00:00 2026-02-20T11:33:23.967 INFO:teuthology.orchestra.run.trial127.stdout:(60/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 757 kB/s | 551 kB 00:00 2026-02-20T11:33:24.047 INFO:teuthology.orchestra.run.trial127.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 371 kB/s | 29 kB 00:00 2026-02-20T11:33:24.129 INFO:teuthology.orchestra.run.trial127.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 731 kB/s | 60 kB 00:00 2026-02-20T11:33:24.185 INFO:teuthology.orchestra.run.trial127.stdout:(63/96): python3-asyncssh-2.13.2-5.el9.noarch.r 2.3 MB/s | 548 kB 00:00 2026-02-20T11:33:24.209 INFO:teuthology.orchestra.run.trial127.stdout:(64/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 540 kB/s | 43 kB 00:00 2026-02-20T11:33:24.260 INFO:teuthology.orchestra.run.trial127.stdout:(65/96): python3-cachetools-4.2.4-1.el9.noarch. 430 kB/s | 32 kB 00:00 2026-02-20T11:33:24.287 INFO:teuthology.orchestra.run.trial127.stdout:(66/96): python3-certifi-2023.05.07-4.el9.noarc 182 kB/s | 14 kB 00:00 2026-02-20T11:33:24.347 INFO:teuthology.orchestra.run.trial127.stdout:(67/96): python3-cheroot-10.0.1-4.el9.noarch.rp 1.9 MB/s | 173 kB 00:00 2026-02-20T11:33:24.398 INFO:teuthology.orchestra.run.trial127.stdout:(68/96): python3-cherrypy-18.6.1-2.el9.noarch.r 3.2 MB/s | 358 kB 00:00 2026-02-20T11:33:24.440 INFO:teuthology.orchestra.run.trial127.stdout:(69/96): python3-google-auth-2.45.0-1.el9.noarc 2.7 MB/s | 254 kB 00:00 2026-02-20T11:33:24.525 INFO:teuthology.orchestra.run.trial127.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 1.7 MB/s | 144 kB 00:00 2026-02-20T11:33:24.609 INFO:teuthology.orchestra.run.trial127.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 1.6 MB/s | 139 kB 00:00 2026-02-20T11:33:24.643 INFO:teuthology.orchestra.run.trial127.stdout:(72/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.3 MB/s | 2.0 MB 00:00 2026-02-20T11:33:24.686 INFO:teuthology.orchestra.run.trial127.stdout:(73/96): python3-isodate-0.6.1-3.el9.noarch.rpm 731 kB/s | 56 kB 00:00 2026-02-20T11:33:24.720 INFO:teuthology.orchestra.run.trial127.stdout:(74/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 138 kB/s | 11 kB 00:00 2026-02-20T11:33:24.760 INFO:teuthology.orchestra.run.trial127.stdout:(75/96): python3-jaraco-classes-3.2.1-5.el9.noa 239 kB/s | 18 kB 00:00 2026-02-20T11:33:24.798 INFO:teuthology.orchestra.run.trial127.stdout:(76/96): python3-jaraco-collections-3.0.0-8.el9 299 kB/s | 23 kB 00:00 2026-02-20T11:33:24.834 INFO:teuthology.orchestra.run.trial127.stdout:(77/96): python3-jaraco-context-6.0.1-3.el9.noa 265 kB/s | 20 kB 00:00 2026-02-20T11:33:24.875 INFO:teuthology.orchestra.run.trial127.stdout:(78/96): python3-jaraco-functools-3.5.0-2.el9.n 251 kB/s | 19 kB 00:00 2026-02-20T11:33:24.909 INFO:teuthology.orchestra.run.trial127.stdout:(79/96): python3-jaraco-text-4.0.0-2.el9.noarch 353 kB/s | 26 kB 00:00 2026-02-20T11:33:24.978 INFO:teuthology.orchestra.run.trial127.stdout:(80/96): python3-kubernetes-26.1.0-3.el9.noarch 10 MB/s | 1.0 MB 00:00 2026-02-20T11:33:24.988 INFO:teuthology.orchestra.run.trial127.stdout:(81/96): python3-more-itertools-8.12.0-2.el9.no 997 kB/s | 79 kB 00:00 2026-02-20T11:33:25.057 INFO:teuthology.orchestra.run.trial127.stdout:(82/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 1.1 MB/s | 86 kB 00:00 2026-02-20T11:33:25.065 INFO:teuthology.orchestra.run.trial127.stdout:(83/96): python3-natsort-7.1.1-5.el9.noarch.rpm 744 kB/s | 58 kB 00:00 2026-02-20T11:33:25.134 INFO:teuthology.orchestra.run.trial127.stdout:(84/96): python3-portend-3.1.0-2.el9.noarch.rpm 212 kB/s | 16 kB 00:00 2026-02-20T11:33:25.146 INFO:teuthology.orchestra.run.trial127.stdout:(85/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.1 MB/s | 90 kB 00:00 2026-02-20T11:33:25.211 INFO:teuthology.orchestra.run.trial127.stdout:(86/96): python3-repoze-lru-0.7-16.el9.noarch.r 397 kB/s | 31 kB 00:00 2026-02-20T11:33:25.234 INFO:teuthology.orchestra.run.trial127.stdout:(87/96): python3-routes-2.5.1-5.el9.noarch.rpm 2.1 MB/s | 188 kB 00:00 2026-02-20T11:33:25.290 INFO:teuthology.orchestra.run.trial127.stdout:(88/96): python3-rsa-4.9-2.el9.noarch.rpm 754 kB/s | 59 kB 00:00 2026-02-20T11:33:25.318 INFO:teuthology.orchestra.run.trial127.stdout:(89/96): python3-saml-1.16.0-1.el9.noarch.rpm 1.5 MB/s | 125 kB 00:00 2026-02-20T11:33:25.368 INFO:teuthology.orchestra.run.trial127.stdout:(90/96): python3-tempora-5.0.0-2.el9.noarch.rpm 462 kB/s | 36 kB 00:00 2026-02-20T11:33:25.398 INFO:teuthology.orchestra.run.trial127.stdout:(91/96): python3-typing-extensions-4.15.0-1.el9 1.1 MB/s | 86 kB 00:00 2026-02-20T11:33:25.447 INFO:teuthology.orchestra.run.trial127.stdout:(92/96): python3-websocket-client-1.2.3-2.el9.n 1.1 MB/s | 90 kB 00:00 2026-02-20T11:33:25.475 INFO:teuthology.orchestra.run.trial127.stdout:(93/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 631 kB/s | 48 kB 00:00 2026-02-20T11:33:25.524 INFO:teuthology.orchestra.run.trial127.stdout:(94/96): python3-xmltodict-0.12.0-15.el9.noarch 286 kB/s | 22 kB 00:00 2026-02-20T11:33:25.549 INFO:teuthology.orchestra.run.trial127.stdout:(95/96): python3-zc-lockfile-2.0-10.el9.noarch. 270 kB/s | 20 kB 00:00 2026-02-20T11:33:33.547 INFO:teuthology.orchestra.run.trial127.stdout:(96/96): ceph-mgr-dashboard-20.3.0-5357.g0c3148 2.3 MB/s | 51 MB 00:22 2026-02-20T11:33:33.549 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:33:33.550 INFO:teuthology.orchestra.run.trial127.stdout:Total 9.3 MB/s | 226 MB 00:24 2026-02-20T11:33:33.951 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:33:33.988 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:33:33.988 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:33:34.789 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:33:34.789 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:33:35.627 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:33:35.629 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-20T11:33:35.633 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-20T11:33:35.654 INFO:teuthology.orchestra.run.trial127.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-20T11:33:35.688 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-20T11:33:35.695 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-20T11:33:35.697 INFO:teuthology.orchestra.run.trial127.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-20T11:33:35.702 INFO:teuthology.orchestra.run.trial127.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-20T11:33:35.725 INFO:teuthology.orchestra.run.trial127.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-20T11:33:35.737 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-20T11:33:35.740 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-20T11:33:35.748 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-20T11:33:35.749 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-20T11:33:35.761 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-20T11:33:35.767 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-20T11:33:35.790 INFO:teuthology.orchestra.run.trial127.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-20T11:33:35.793 INFO:teuthology.orchestra.run.trial127.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-20T11:33:35.843 INFO:teuthology.orchestra.run.trial127.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-20T11:33:35.855 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-20T11:33:35.858 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-20T11:33:35.862 INFO:teuthology.orchestra.run.trial127.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-20T11:33:35.867 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-20T11:33:35.872 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-20T11:33:35.875 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-20T11:33:35.889 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-20T11:33:35.894 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-20T11:33:35.899 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-20T11:33:35.910 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-20T11:33:35.919 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-20T11:33:35.925 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-20T11:33:35.946 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-20T11:33:35.957 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-20T11:33:35.972 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-20T11:33:35.977 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-20T11:33:35.984 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-20T11:33:36.024 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-20T11:33:36.344 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-20T11:33:36.354 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-20T11:33:36.357 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-20T11:33:36.363 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-20T11:33:36.365 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-20T11:33:36.392 INFO:teuthology.orchestra.run.trial127.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-20T11:33:36.435 INFO:teuthology.orchestra.run.trial127.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-20T11:33:36.444 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-20T11:33:36.449 INFO:teuthology.orchestra.run.trial127.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-20T11:33:36.454 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-20T11:33:36.460 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-20T11:33:36.464 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-20T11:33:36.471 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-20T11:33:36.475 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-20T11:33:36.503 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-20T11:33:36.514 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-20T11:33:36.520 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-20T11:33:36.527 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-20T11:33:36.561 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-20T11:33:36.782 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-20T11:33:36.805 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-20T11:33:36.809 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-20T11:33:36.867 INFO:teuthology.orchestra.run.trial127.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-20T11:33:36.868 INFO:teuthology.orchestra.run.trial127.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-20T11:33:36.889 INFO:teuthology.orchestra.run.trial127.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-20T11:33:37.210 INFO:teuthology.orchestra.run.trial127.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-20T11:33:37.281 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-20T11:33:37.943 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-20T11:33:37.959 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-20T11:33:37.969 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-20T11:33:37.976 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-20T11:33:37.989 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-20T11:33:38.004 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-20T11:33:38.079 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-20T11:33:38.087 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-20T11:33:38.111 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-20T11:33:38.140 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-20T11:33:38.190 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-20T11:33:38.196 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-20T11:33:38.199 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-20T11:33:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-20T11:33:38.219 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:33:38.219 INFO:teuthology.orchestra.run.trial127.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-20T11:33:38.219 INFO:teuthology.orchestra.run.trial127.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-20T11:33:38.219 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:38.226 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:33:38.252 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:33:38.252 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-20T11:33:38.252 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:38.288 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-20T11:33:38.876 INFO:teuthology.orchestra.run.trial127.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-20T11:33:44.272 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 80/96 2026-02-20T11:33:44.336 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 81/96 2026-02-20T11:33:44.338 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 82/96 2026-02-20T11:33:44.352 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 82/96 2026-02-20T11:33:44.352 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:33:44.353 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-20T11:33:44.353 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:44.363 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 83/96 2026-02-20T11:33:44.387 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 83/96 2026-02-20T11:33:44.387 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:33:44.387 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-20T11:33:44.387 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T11:33:44.387 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T11:33:44.387 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:44.446 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 84/96 2026-02-20T11:33:44.448 INFO:teuthology.orchestra.run.trial127.stdout: Installing : cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 84/96 2026-02-20T11:33:44.457 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9 85/96 2026-02-20T11:33:44.480 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el 86/96 2026-02-20T11:33:44.482 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 87/96 2026-02-20T11:33:47.376 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 87/96 2026-02-20T11:33:47.382 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 88/96 2026-02-20T11:33:47.835 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 88/96 2026-02-20T11:33:47.836 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 89/96 2026-02-20T11:33:47.848 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 89/96 2026-02-20T11:33:47.849 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 90/96 2026-02-20T11:33:47.911 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 90/96 2026-02-20T11:33:47.951 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9. 91/96 2026-02-20T11:33:47.952 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 92/96 2026-02-20T11:33:47.974 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 92/96 2026-02-20T11:33:47.974 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:33:47.974 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-20T11:33:47.974 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T11:33:47.974 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T11:33:47.974 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:47.987 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 93/96 2026-02-20T11:33:48.202 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 93/96 2026-02-20T11:33:48.203 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 94/96 2026-02-20T11:33:48.224 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 94/96 2026-02-20T11:33:48.224 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:33:48.224 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-20T11:33:48.224 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T11:33:48.224 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T11:33:48.225 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:48.306 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 95/96 2026-02-20T11:33:48.326 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 95/96 2026-02-20T11:33:48.326 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:33:48.326 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-20T11:33:48.326 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T11:33:48.326 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T11:33:48.326 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:48.327 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 96/96 2026-02-20T11:33:48.334 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 96/96 2026-02-20T11:33:48.358 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 96/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 96/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 4/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 5/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 6/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 7/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el 8/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 9/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 10/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 11/96 2026-02-20T11:33:49.740 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 12/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9. 13/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 14/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9 15/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 16/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 17/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-20T11:33:49.741 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-20T11:33:49.742 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-20T11:33:49.743 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-20T11:33:49.807 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: lua-5.4.4-4.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-20T11:33:49.808 INFO:teuthology.orchestra.run.trial127.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-ply-3.11-14.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-20T11:33:49.809 INFO:teuthology.orchestra.run.trial127.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: unzip-6.0-59.el9.x86_64 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: zip-3.0-35.el9.x86_64 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:49.810 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:49.880 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-base 2026-02-20T11:33:50.198 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:02 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:50.262 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:33:50.281 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:50.282 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:33:50.282 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:50.304 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install cephadm 2026-02-20T11:33:50.612 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:02 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:50.678 INFO:teuthology.orchestra.run.trial127.stdout:Package cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:33:50.696 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:50.697 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:33:50.697 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:50.719 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T11:33:51.026 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:03 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:51.112 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:51.112 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:51.112 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repo Size 2026-02-20T11:33:51.113 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:51.113 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-20T11:33:51.113 INFO:teuthology.orchestra.run.trial127.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 149 k 2026-02-20T11:33:51.113 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:51.113 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:33:51.113 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:51.113 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-20T11:33:51.113 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:51.113 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 149 k 2026-02-20T11:33:51.113 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 455 k 2026-02-20T11:33:51.114 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:33:51.586 INFO:teuthology.orchestra.run.trial127.stdout:ceph-immutable-object-cache-20.3.0-5357.g0c3148 316 kB/s | 149 kB 00:00 2026-02-20T11:33:51.586 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:33:51.586 INFO:teuthology.orchestra.run.trial127.stdout:Total 315 kB/s | 149 kB 00:00 2026-02-20T11:33:51.586 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:33:51.591 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:33:51.591 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:33:51.622 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:33:51.622 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:33:51.677 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:33:51.678 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:33:51.701 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:33:51.701 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:33:51.701 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-20T11:33:51.701 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:51.981 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:33:51.981 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:51.981 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-20T11:33:51.982 INFO:teuthology.orchestra.run.trial127.stdout: ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:51.982 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:51.982 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:52.009 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr 2026-02-20T11:33:52.321 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:04 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:52.387 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:33:52.406 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:52.406 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:33:52.406 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:52.429 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T11:33:52.737 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:04 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:52.803 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:33:52.821 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:52.821 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:33:52.822 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:52.844 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T11:33:53.150 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:05 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:53.218 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:33:53.237 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:53.237 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:33:53.237 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:53.259 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-rook 2026-02-20T11:33:53.568 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:05 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:53.638 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:33:53.656 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:53.657 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:33:53.657 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:53.679 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T11:33:53.996 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:05 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:54.061 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:33:54.080 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:54.081 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:33:54.081 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:54.102 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-osd 2026-02-20T11:33:54.407 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:06 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:54.472 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:33:54.490 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:54.491 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:33:54.491 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:54.512 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-osd-classic 2026-02-20T11:33:54.825 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:06 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:54.889 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:33:54.907 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:54.908 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:33:54.908 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:54.929 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-fuse 2026-02-20T11:33:55.239 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:07 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:55.324 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:55.324 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:55.324 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repo Size 2026-02-20T11:33:55.324 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:55.324 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-20T11:33:55.324 INFO:teuthology.orchestra.run.trial127.stdout: ceph-fuse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 914 k 2026-02-20T11:33:55.324 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:55.325 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:33:55.325 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:55.325 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-20T11:33:55.325 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:55.325 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 914 k 2026-02-20T11:33:55.325 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 2.9 M 2026-02-20T11:33:55.325 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:33:56.542 INFO:teuthology.orchestra.run.trial127.stdout:ceph-fuse-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 751 kB/s | 914 kB 00:01 2026-02-20T11:33:56.544 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:33:56.544 INFO:teuthology.orchestra.run.trial127.stdout:Total 750 kB/s | 914 kB 00:01 2026-02-20T11:33:56.544 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:33:56.548 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:33:56.549 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:33:56.582 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:33:56.582 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:33:56.672 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:33:56.689 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:33:56.940 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:33:56.996 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:33:56.997 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:56.997 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-20T11:33:56.997 INFO:teuthology.orchestra.run.trial127.stdout: ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:56.997 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:56.997 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:57.024 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-volume 2026-02-20T11:33:57.335 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:09 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:57.402 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:33:57.421 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:57.421 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:33:57.421 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:57.443 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install librados-devel 2026-02-20T11:33:57.754 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:09 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:57.838 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:57.838 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:57.838 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repo Size 2026-02-20T11:33:57.838 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:57.838 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-20T11:33:57.839 INFO:teuthology.orchestra.run.trial127.stdout: librados-devel x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 124 k 2026-02-20T11:33:57.839 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:57.839 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:33:57.839 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:57.839 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-20T11:33:57.839 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:57.839 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 124 k 2026-02-20T11:33:57.839 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 434 k 2026-02-20T11:33:57.839 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:33:58.305 INFO:teuthology.orchestra.run.trial127.stdout:librados-devel-20.3.0-5357.g0c3148ca.el9.x86_64 266 kB/s | 124 kB 00:00 2026-02-20T11:33:58.305 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:33:58.305 INFO:teuthology.orchestra.run.trial127.stdout:Total 265 kB/s | 124 kB 00:00 2026-02-20T11:33:58.305 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:33:58.308 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:33:58.308 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:33:58.343 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:33:58.343 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:33:58.398 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:33:58.415 INFO:teuthology.orchestra.run.trial127.stdout: Installing : librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:33:58.492 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:33:58.549 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:33:58.549 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:58.549 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-20T11:33:58.550 INFO:teuthology.orchestra.run.trial127.stdout: librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:33:58.550 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:58.550 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:58.578 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install libcephfs2 2026-02-20T11:33:58.907 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:10 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:58.972 INFO:teuthology.orchestra.run.trial127.stdout:Package libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:33:58.991 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:58.991 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:33:58.992 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:33:59.013 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install libcephfs-devel 2026-02-20T11:33:59.323 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:11 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:33:59.407 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:33:59.407 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:59.407 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repo Size 2026-02-20T11:33:59.408 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:59.408 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-20T11:33:59.408 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-devel x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 37 k 2026-02-20T11:33:59.408 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:59.408 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:33:59.408 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:33:59.408 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-20T11:33:59.408 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:33:59.408 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 37 k 2026-02-20T11:33:59.408 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 171 k 2026-02-20T11:33:59.408 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:33:59.924 INFO:teuthology.orchestra.run.trial127.stdout:libcephfs-devel-20.3.0-5357.g0c3148ca.el9.x86_6 72 kB/s | 37 kB 00:00 2026-02-20T11:33:59.925 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:33:59.925 INFO:teuthology.orchestra.run.trial127.stdout:Total 72 kB/s | 37 kB 00:00 2026-02-20T11:33:59.925 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:33:59.927 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:33:59.928 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:33:59.930 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:33:59.930 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:33:59.952 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:33:59.968 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:00.023 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:00.080 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:00.080 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:00.080 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-20T11:34:00.080 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:34:00.080 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:00.080 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:34:00.108 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install librados2 2026-02-20T11:34:00.423 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:12 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:34:00.490 INFO:teuthology.orchestra.run.trial127.stdout:Package librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:34:00.508 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:34:00.509 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:34:00.509 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:34:00.531 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install librbd1 2026-02-20T11:34:00.833 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:12 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:34:00.898 INFO:teuthology.orchestra.run.trial127.stdout:Package librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:34:00.917 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:34:00.918 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:34:00.918 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:34:00.939 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-rados 2026-02-20T11:34:01.247 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:13 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:34:01.314 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:34:01.332 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:34:01.333 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:34:01.333 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:34:01.355 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-rgw 2026-02-20T11:34:01.668 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:13 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:34:01.735 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:34:01.753 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:34:01.754 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:34:01.754 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:34:01.775 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-cephfs 2026-02-20T11:34:02.090 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:14 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:34:02.154 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:34:02.173 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:34:02.174 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:34:02.174 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:34:02.196 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-rbd 2026-02-20T11:34:02.506 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:14 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:34:02.572 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:34:02.590 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:34:02.591 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:34:02.591 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:34:02.613 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install rbd-fuse 2026-02-20T11:34:02.921 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:14 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:34:03.005 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:34:03.006 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:03.006 INFO:teuthology.orchestra.run.trial127.stdout: Package Architecture Version Repository Size 2026-02-20T11:34:03.006 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:03.006 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-20T11:34:03.006 INFO:teuthology.orchestra.run.trial127.stdout: rbd-fuse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 86 k 2026-02-20T11:34:03.006 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:03.006 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:34:03.006 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:03.006 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-20T11:34:03.007 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:03.007 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 86 k 2026-02-20T11:34:03.007 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 236 k 2026-02-20T11:34:03.007 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:34:03.418 INFO:teuthology.orchestra.run.trial127.stdout:rbd-fuse-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 210 kB/s | 86 kB 00:00 2026-02-20T11:34:03.419 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:34:03.419 INFO:teuthology.orchestra.run.trial127.stdout:Total 209 kB/s | 86 kB 00:00 2026-02-20T11:34:03.419 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:34:03.423 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:34:03.423 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:34:03.454 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:34:03.454 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:34:03.506 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:34:03.522 INFO:teuthology.orchestra.run.trial127.stdout: Installing : rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:03.600 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:03.657 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:03.658 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:03.658 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-20T11:34:03.658 INFO:teuthology.orchestra.run.trial127.stdout: rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:34:03.658 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:03.658 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:34:03.686 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install rbd-mirror 2026-02-20T11:34:04.001 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:15 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:34:04.086 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:34:04.086 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:04.086 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repo Size 2026-02-20T11:34:04.086 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:04.086 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-20T11:34:04.086 INFO:teuthology.orchestra.run.trial127.stdout: rbd-mirror x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.6 M 2026-02-20T11:34:04.086 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:04.087 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:34:04.087 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:04.087 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-20T11:34:04.087 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:04.087 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 2.6 M 2026-02-20T11:34:04.087 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 11 M 2026-02-20T11:34:04.087 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:34:06.446 INFO:teuthology.orchestra.run.trial127.stdout:rbd-mirror-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 1.1 MB/s | 2.6 MB 00:02 2026-02-20T11:34:06.446 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:34:06.446 INFO:teuthology.orchestra.run.trial127.stdout:Total 1.1 MB/s | 2.6 MB 00:02 2026-02-20T11:34:06.446 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:34:06.451 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:34:06.451 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:34:06.491 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:34:06.491 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:34:06.654 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:34:06.655 INFO:teuthology.orchestra.run.trial127.stdout: Installing : rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:06.679 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:06.679 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:34:06.680 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-20T11:34:06.680 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T11:34:06.680 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T11:34:06.680 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:06.998 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:06.999 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:06.999 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-20T11:34:06.999 INFO:teuthology.orchestra.run.trial127.stdout: rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:34:06.999 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:06.999 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:34:07.028 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install rbd-nbd 2026-02-20T11:34:07.339 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:19 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:34:07.425 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:34:07.425 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:07.425 INFO:teuthology.orchestra.run.trial127.stdout: Package Architecture Version Repository Size 2026-02-20T11:34:07.425 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:07.425 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-20T11:34:07.425 INFO:teuthology.orchestra.run.trial127.stdout: rbd-nbd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 167 k 2026-02-20T11:34:07.426 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:07.426 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:34:07.426 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:07.426 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-20T11:34:07.426 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:07.426 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 167 k 2026-02-20T11:34:07.426 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 496 k 2026-02-20T11:34:07.426 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:34:07.835 INFO:teuthology.orchestra.run.trial127.stdout:rbd-nbd-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 407 kB/s | 167 kB 00:00 2026-02-20T11:34:07.835 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:34:07.835 INFO:teuthology.orchestra.run.trial127.stdout:Total 406 kB/s | 167 kB 00:00 2026-02-20T11:34:07.835 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:34:07.840 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:34:07.840 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:34:07.873 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:34:07.874 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:34:07.931 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:34:07.948 INFO:teuthology.orchestra.run.trial127.stdout: Installing : rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:08.029 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:08.087 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:34:08.087 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:08.087 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-20T11:34:08.087 INFO:teuthology.orchestra.run.trial127.stdout: rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:34:08.087 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:08.087 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:34:08.116 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-osd-crimson 2026-02-20T11:34:08.427 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:20 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:34:08.491 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:34:08.510 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:34:08.511 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:34:08.511 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:34:08.532 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-20T11:34:08.847 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:03:20 ago on Fri 20 Feb 2026 11:30:48 AM UTC. 2026-02-20T11:34:08.931 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:34:08.932 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:08.932 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repo Size 2026-02-20T11:34:08.932 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:08.932 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-20T11:34:08.932 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 433 M 2026-02-20T11:34:08.932 INFO:teuthology.orchestra.run.trial127.stdout:Installing weak dependencies: 2026-02-20T11:34:08.932 INFO:teuthology.orchestra.run.trial127.stdout: ceph-debugsource x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 12 M 2026-02-20T11:34:08.932 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:08.932 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-20T11:34:08.933 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-20T11:34:08.933 INFO:teuthology.orchestra.run.trial127.stdout:Install 2 Packages 2026-02-20T11:34:08.933 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:34:08.933 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 445 M 2026-02-20T11:34:08.933 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 2.7 G 2026-02-20T11:34:08.933 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-20T11:34:23.502 INFO:teuthology.orchestra.run.trial127.stdout:(1/2): ceph-debugsource-20.3.0-5357.g0c3148ca.e 846 kB/s | 12 MB 00:14 2026-02-20T11:37:44.583 INFO:teuthology.orchestra.run.trial127.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5357.g 2.0 MB/s | 433 MB 03:35 2026-02-20T11:37:44.584 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:37:44.584 INFO:teuthology.orchestra.run.trial127.stdout:Total 2.1 MB/s | 445 MB 03:35 2026-02-20T11:37:44.584 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-20T11:37:44.590 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-20T11:37:44.590 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-20T11:37:45.959 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-20T11:37:45.959 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-20T11:37:49.570 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-20T11:38:11.813 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/2 2026-02-20T11:38:11.830 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:38:13.842 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:38:13.842 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/2 2026-02-20T11:38:13.900 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:38:13.900 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:13.901 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-20T11:38:13.901 INFO:teuthology.orchestra.run.trial127.stdout: ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:38:13.901 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:38:13.901 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:13.901 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:13.979 DEBUG:teuthology.parallel:result is None 2026-02-20T11:38:13.979 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:38:14.148 DEBUG:teuthology.orchestra.run.trial127:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T11:38:14.168 INFO:teuthology.orchestra.run.trial127.stdout:20.3.0-5357.g0c3148ca.el9 2026-02-20T11:38:14.169 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5357.g0c3148ca.el9 2026-02-20T11:38:14.169 INFO:teuthology.task.install:The correct ceph version 20.3.0-5357.g0c3148ca is installed. 2026-02-20T11:38:14.171 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-20T11:38:14.171 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:38:14.171 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-20T11:38:14.235 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-20T11:38:14.236 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:38:14.236 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/usr/bin/daemon-helper 2026-02-20T11:38:14.301 DEBUG:teuthology.orchestra.run.trial127:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-20T11:38:14.365 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-20T11:38:14.366 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:38:14.366 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-20T11:38:14.429 DEBUG:teuthology.orchestra.run.trial127:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-20T11:38:14.494 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-20T11:38:14.494 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:38:14.495 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/usr/bin/stdin-killer 2026-02-20T11:38:14.559 DEBUG:teuthology.orchestra.run.trial127:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-20T11:38:14.622 INFO:teuthology.run_tasks:Running task pexec... 2026-02-20T11:38:14.637 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-20T11:38:14.637 DEBUG:teuthology.orchestra.run.trial127:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-20T11:38:14.666 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial127.front.sepia.ceph.com 2026-02-20T11:38:14.667 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-20T11:38:14.716 DEBUG:teuthology.parallel:result is None 2026-02-20T11:38:14.716 INFO:teuthology.run_tasks:Running task install... 2026-02-20T11:38:14.727 DEBUG:teuthology.task.install:project ceph 2026-02-20T11:38:14.727 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}} 2026-02-20T11:38:14.727 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'} 2026-02-20T11:38:14.727 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T11:38:14.730 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T11:38:14.730 INFO:teuthology.task.install:extra packages: [] 2026-02-20T11:38:14.730 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'tag': None, 'wait_for_package': False} 2026-02-20T11:38:14.731 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:38:14.863 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/ 2026-02-20T11:38:14.863 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5357.g0c3148ca 2026-02-20T11:38:15.211 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T11:38:15.212 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:38:15.212 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T11:38:15.236 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-20T11:38:15.237 DEBUG:teuthology.orchestra.run.trial127:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T11:38:15.303 DEBUG:teuthology.orchestra.run.trial127:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T11:38:15.369 DEBUG:teuthology.orchestra.run.trial127:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T11:38:15.427 INFO:teuthology.orchestra.run.trial127.stdout:check_obsoletes = 1 2026-02-20T11:38:15.437 DEBUG:teuthology.orchestra.run.trial127:> sudo yum clean all 2026-02-20T11:38:15.600 INFO:teuthology.orchestra.run.trial127.stdout:56 files removed 2026-02-20T11:38:15.616 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-radosgw 2026-02-20T11:38:16.455 INFO:teuthology.orchestra.run.trial127.stdout:ceph packages for x86_64 137 kB/s | 94 kB 00:00 2026-02-20T11:38:17.090 INFO:teuthology.orchestra.run.trial127.stdout:ceph noarch packages 40 kB/s | 24 kB 00:00 2026-02-20T11:38:17.685 INFO:teuthology.orchestra.run.trial127.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-20T11:38:22.174 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.9 MB 00:04 2026-02-20T11:38:25.955 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 27 MB 00:03 2026-02-20T11:38:29.750 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 7.8 MB 00:01 2026-02-20T11:38:31.654 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 20 kB 00:01 2026-02-20T11:38:32.022 INFO:teuthology.orchestra.run.trial127.stdout:Extra Packages for Enterprise Linux 66 MB/s | 20 MB 00:00 2026-02-20T11:38:35.366 INFO:teuthology.orchestra.run.trial127.stdout:lab-extras 295 kB/s | 50 kB 00:00 2026-02-20T11:38:36.544 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:36.562 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:36.563 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:36.563 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:36.586 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-test 2026-02-20T11:38:36.891 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:36.956 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:36.975 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:36.976 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:36.976 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:36.997 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph 2026-02-20T11:38:37.310 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:37.376 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:37.394 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:37.395 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:37.395 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:37.416 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-base 2026-02-20T11:38:37.725 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:37.789 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:37.808 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:37.809 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:37.809 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:37.830 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install cephadm 2026-02-20T11:38:38.143 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:38.210 INFO:teuthology.orchestra.run.trial127.stdout:Package cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:38:38.228 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:38.229 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:38.229 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:38.251 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T11:38:38.567 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:38.632 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:38.650 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:38.651 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:38.651 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:38.673 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr 2026-02-20T11:38:38.986 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:39.051 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:39.069 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:39.070 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:39.070 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:39.092 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T11:38:39.407 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:39.472 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:38:39.490 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:39.491 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:39.491 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:39.513 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T11:38:39.829 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:39.896 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:38:39.915 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:39.915 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:39.916 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:39.938 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-rook 2026-02-20T11:38:40.247 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:05 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:40.312 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:38:40.331 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:40.332 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:40.332 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:40.354 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T11:38:40.661 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:05 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:40.726 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:38:40.744 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:40.745 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:40.745 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:40.767 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-osd 2026-02-20T11:38:41.082 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:41.147 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:41.165 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:41.166 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:41.166 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:41.189 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-osd-classic 2026-02-20T11:38:41.500 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:41.567 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:41.586 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:41.587 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:41.587 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:41.608 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-fuse 2026-02-20T11:38:41.920 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:41.985 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:42.003 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:42.004 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:42.004 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:42.026 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-volume 2026-02-20T11:38:42.342 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:42.407 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:38:42.425 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:42.427 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:42.427 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:42.448 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install librados-devel 2026-02-20T11:38:42.761 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:42.826 INFO:teuthology.orchestra.run.trial127.stdout:Package librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:42.844 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:42.845 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:42.845 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:42.867 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install libcephfs2 2026-02-20T11:38:43.184 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:08 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:43.249 INFO:teuthology.orchestra.run.trial127.stdout:Package libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:43.268 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:43.268 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:43.269 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:43.290 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install libcephfs-devel 2026-02-20T11:38:43.606 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:08 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:43.673 INFO:teuthology.orchestra.run.trial127.stdout:Package libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:43.692 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:43.693 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:43.693 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:43.715 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install librados2 2026-02-20T11:38:44.023 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:44.088 INFO:teuthology.orchestra.run.trial127.stdout:Package librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:44.107 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:44.107 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:44.107 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:44.130 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install librbd1 2026-02-20T11:38:44.435 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:44.500 INFO:teuthology.orchestra.run.trial127.stdout:Package librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:44.518 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:44.519 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:44.519 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:44.541 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-rados 2026-02-20T11:38:44.855 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:44.922 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:44.941 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:44.941 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:44.942 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:44.964 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-rgw 2026-02-20T11:38:45.276 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:10 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:45.341 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:45.359 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:45.360 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:45.360 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:45.383 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-cephfs 2026-02-20T11:38:45.695 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:10 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:45.762 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:45.781 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:45.782 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:45.782 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:45.803 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-rbd 2026-02-20T11:38:46.117 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:46.184 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:46.202 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:46.203 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:46.203 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:46.225 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install rbd-fuse 2026-02-20T11:38:46.539 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:46.604 INFO:teuthology.orchestra.run.trial127.stdout:Package rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:46.622 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:46.623 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:46.623 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:46.645 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install rbd-mirror 2026-02-20T11:38:46.953 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:47.019 INFO:teuthology.orchestra.run.trial127.stdout:Package rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:47.038 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:47.039 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:47.039 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:47.061 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install rbd-nbd 2026-02-20T11:38:47.369 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:12 ago on Fri 20 Feb 2026 11:38:35 AM UTC. 2026-02-20T11:38:47.435 INFO:teuthology.orchestra.run.trial127.stdout:Package rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:38:47.454 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-20T11:38:47.455 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-20T11:38:47.455 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-20T11:38:47.480 DEBUG:teuthology.parallel:result is None 2026-02-20T11:38:47.480 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:38:47.731 DEBUG:teuthology.orchestra.run.trial127:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T11:38:47.752 INFO:teuthology.orchestra.run.trial127.stdout:20.3.0-5357.g0c3148ca.el9 2026-02-20T11:38:47.752 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5357.g0c3148ca.el9 2026-02-20T11:38:47.752 INFO:teuthology.task.install:The correct ceph version 20.3.0-5357.g0c3148ca is installed. 2026-02-20T11:38:47.755 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-20T11:38:47.755 INFO:teuthology.run_tasks:Running task ceph... 2026-02-20T11:38:47.848 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-20T11:38:47.848 DEBUG:teuthology.orchestra.run.trial127:> sudo chmod 777 /var/log/ceph 2026-02-20T11:38:47.873 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-20T11:38:47.873 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-20T11:38:47.938 INFO:tasks.ceph:Creating extra log directories... 2026-02-20T11:38:47.939 DEBUG:teuthology.orchestra.run.trial127:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-20T11:38:48.009 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-20T11:38:48.009 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-20T11:38:48.010 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60490', 'branch': 'wip-perezjos-crimson-only-19-02-2026-PR64439', 'description': 'crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rbd_cls_tests}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '60490', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-perezjos-crimson-only-19-02-2026-PR64439'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '445d5c9ba18a4040e70e25303230c36c77773acb'}}, 'owner': 'scheduled_jjperez@soko04.front.sepia.ceph.com', 'priority': 101, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 1736, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '445d5c9ba18a4040e70e25303230c36c77773acb', 'targets': {'trial127.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCfbDtBOKvQP5IdSL2H5iB3GhHqW3owcH1ZPa8tpvf4XdZLs2vELWgT9Hs0209l3MQdgxsrX+tMIH5OMUALxp18='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_rbd.sh', 'cls/test_cls_lock.sh', 'cls/test_cls_journal.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'reimage-ssh-down', 'teuthology_sha1': 'e6d9099ce03dc21193fda653159ec82465c3a94a', 'timestamp': '2026-02-20_10:21:31', 'tube': 'trial', 'user': 'jjperez', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-20T11:38:48.010 DEBUG:teuthology.orchestra.run.trial127:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-20T11:38:48.058 DEBUG:teuthology.orchestra.run.trial127:> sudo install -d -m0777 -- /var/run/ceph 2026-02-20T11:38:48.124 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:38:48.124 DEBUG:teuthology.orchestra.run.trial127:> dd if=/scratch_devs of=/dev/stdout 2026-02-20T11:38:48.182 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-20T11:38:48.182 DEBUG:teuthology.orchestra.run.trial127:> stat /dev/vg_nvme/lv_1 2026-02-20T11:38:48.239 INFO:teuthology.orchestra.run.trial127.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-20T11:38:48.239 INFO:teuthology.orchestra.run.trial127.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:38:48.239 INFO:teuthology.orchestra.run.trial127.stdout:Device: 6h/6d Inode: 1288 Links: 1 2026-02-20T11:38:48.240 INFO:teuthology.orchestra.run.trial127.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:38:48.240 INFO:teuthology.orchestra.run.trial127.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:38:48.240 INFO:teuthology.orchestra.run.trial127.stdout:Access: 2026-02-20 11:34:06.823560593 +0000 2026-02-20T11:38:48.240 INFO:teuthology.orchestra.run.trial127.stdout:Modify: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.240 INFO:teuthology.orchestra.run.trial127.stdout:Change: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.240 INFO:teuthology.orchestra.run.trial127.stdout: Birth: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.240 DEBUG:teuthology.orchestra.run.trial127:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-20T11:38:48.303 INFO:teuthology.orchestra.run.trial127.stderr:1+0 records in 2026-02-20T11:38:48.303 INFO:teuthology.orchestra.run.trial127.stderr:1+0 records out 2026-02-20T11:38:48.303 INFO:teuthology.orchestra.run.trial127.stderr:512 bytes copied, 0.000183592 s, 2.8 MB/s 2026-02-20T11:38:48.304 DEBUG:teuthology.orchestra.run.trial127:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-20T11:38:48.363 DEBUG:teuthology.orchestra.run.trial127:> stat /dev/vg_nvme/lv_2 2026-02-20T11:38:48.419 INFO:teuthology.orchestra.run.trial127.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-20T11:38:48.419 INFO:teuthology.orchestra.run.trial127.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:38:48.419 INFO:teuthology.orchestra.run.trial127.stdout:Device: 6h/6d Inode: 1284 Links: 1 2026-02-20T11:38:48.420 INFO:teuthology.orchestra.run.trial127.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:38:48.420 INFO:teuthology.orchestra.run.trial127.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:38:48.420 INFO:teuthology.orchestra.run.trial127.stdout:Access: 2026-02-20 11:34:06.823560593 +0000 2026-02-20T11:38:48.420 INFO:teuthology.orchestra.run.trial127.stdout:Modify: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.420 INFO:teuthology.orchestra.run.trial127.stdout:Change: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.420 INFO:teuthology.orchestra.run.trial127.stdout: Birth: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.420 DEBUG:teuthology.orchestra.run.trial127:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-20T11:38:48.482 INFO:teuthology.orchestra.run.trial127.stderr:1+0 records in 2026-02-20T11:38:48.482 INFO:teuthology.orchestra.run.trial127.stderr:1+0 records out 2026-02-20T11:38:48.482 INFO:teuthology.orchestra.run.trial127.stderr:512 bytes copied, 0.000133318 s, 3.8 MB/s 2026-02-20T11:38:48.483 DEBUG:teuthology.orchestra.run.trial127:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-20T11:38:48.540 DEBUG:teuthology.orchestra.run.trial127:> stat /dev/vg_nvme/lv_3 2026-02-20T11:38:48.596 INFO:teuthology.orchestra.run.trial127.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-20T11:38:48.597 INFO:teuthology.orchestra.run.trial127.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:38:48.597 INFO:teuthology.orchestra.run.trial127.stdout:Device: 6h/6d Inode: 1298 Links: 1 2026-02-20T11:38:48.597 INFO:teuthology.orchestra.run.trial127.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:38:48.597 INFO:teuthology.orchestra.run.trial127.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:38:48.597 INFO:teuthology.orchestra.run.trial127.stdout:Access: 2026-02-20 11:34:06.823560593 +0000 2026-02-20T11:38:48.597 INFO:teuthology.orchestra.run.trial127.stdout:Modify: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.597 INFO:teuthology.orchestra.run.trial127.stdout:Change: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.597 INFO:teuthology.orchestra.run.trial127.stdout: Birth: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.598 DEBUG:teuthology.orchestra.run.trial127:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-20T11:38:48.662 INFO:teuthology.orchestra.run.trial127.stderr:1+0 records in 2026-02-20T11:38:48.662 INFO:teuthology.orchestra.run.trial127.stderr:1+0 records out 2026-02-20T11:38:48.663 INFO:teuthology.orchestra.run.trial127.stderr:512 bytes copied, 0.000146162 s, 3.5 MB/s 2026-02-20T11:38:48.664 DEBUG:teuthology.orchestra.run.trial127:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-20T11:38:48.723 DEBUG:teuthology.orchestra.run.trial127:> stat /dev/vg_nvme/lv_4 2026-02-20T11:38:48.780 INFO:teuthology.orchestra.run.trial127.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-20T11:38:48.781 INFO:teuthology.orchestra.run.trial127.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:38:48.781 INFO:teuthology.orchestra.run.trial127.stdout:Device: 6h/6d Inode: 1285 Links: 1 2026-02-20T11:38:48.781 INFO:teuthology.orchestra.run.trial127.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:38:48.781 INFO:teuthology.orchestra.run.trial127.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:38:48.781 INFO:teuthology.orchestra.run.trial127.stdout:Access: 2026-02-20 11:34:06.823560593 +0000 2026-02-20T11:38:48.781 INFO:teuthology.orchestra.run.trial127.stdout:Modify: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.781 INFO:teuthology.orchestra.run.trial127.stdout:Change: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.781 INFO:teuthology.orchestra.run.trial127.stdout: Birth: 2026-02-20 11:31:02.701329891 +0000 2026-02-20T11:38:48.782 DEBUG:teuthology.orchestra.run.trial127:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-20T11:38:48.843 INFO:teuthology.orchestra.run.trial127.stderr:1+0 records in 2026-02-20T11:38:48.843 INFO:teuthology.orchestra.run.trial127.stderr:1+0 records out 2026-02-20T11:38:48.843 INFO:teuthology.orchestra.run.trial127.stderr:512 bytes copied, 0.000140842 s, 3.6 MB/s 2026-02-20T11:38:48.845 DEBUG:teuthology.orchestra.run.trial127:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-20T11:38:48.904 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:38:48.904 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial127.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T11:38:48.904 INFO:tasks.ceph:Generating config... 2026-02-20T11:38:48.906 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-20T11:38:48.906 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-20T11:38:48.906 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-20T11:38:48.907 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-20T11:38:48.907 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-20T11:38:48.907 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-20T11:38:48.907 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-20T11:38:48.907 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-20T11:38:48.907 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-20T11:38:48.907 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-20T11:38:48.907 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-20T11:38:48.907 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-20T11:38:48.907 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-20T11:38:48.907 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-20T11:38:48.908 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-20T11:38:48.909 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-20T11:38:48.909 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-20T11:38:48.909 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-20T11:38:48.909 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-20T11:38:48.909 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-20T11:38:48.909 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-20T11:38:48.909 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-20T11:38:48.909 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-20T11:38:48.909 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-20T11:38:48.909 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-20T11:38:48.909 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-20T11:38:48.909 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-20T11:38:48.910 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-20T11:38:48.910 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-20T11:38:48.910 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-20T11:38:48.910 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-20T11:38:48.910 INFO:tasks.ceph:Setting up mon.a... 2026-02-20T11:38:48.910 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-20T11:38:48.986 INFO:teuthology.orchestra.run.trial127.stdout:creating /etc/ceph/ceph.keyring 2026-02-20T11:38:48.989 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-20T11:38:49.072 DEBUG:teuthology.orchestra.run.trial127:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T11:38:49.140 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.127')] 2026-02-20T11:38:49.141 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.127', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-20T11:38:49.142 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:38:49.142 DEBUG:teuthology.orchestra.run.trial127:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T11:38:49.197 DEBUG:teuthology.orchestra.run.trial127:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.127 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:38:49.276 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:38:49.274+0000 7f11f9666f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:38:49.277 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:38:49.274+0000 7f11f9666f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:38:49.277 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:38:49.274+0000 7f11f9666f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:38:49.278 INFO:teuthology.orchestra.run.trial127.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:38:49.279 INFO:teuthology.orchestra.run.trial127.stdout:monmaptool: generated fsid f7dd0554-3a23-4321-9783-f9664d26aacc 2026-02-20T11:38:49.279 INFO:teuthology.orchestra.run.trial127.stdout:setting min_mon_release = tentacle 2026-02-20T11:38:49.279 INFO:teuthology.orchestra.run.trial127.stdout:epoch 0 2026-02-20T11:38:49.279 INFO:teuthology.orchestra.run.trial127.stdout:fsid f7dd0554-3a23-4321-9783-f9664d26aacc 2026-02-20T11:38:49.279 INFO:teuthology.orchestra.run.trial127.stdout:last_changed 2026-02-20T11:38:49.275524+0000 2026-02-20T11:38:49.279 INFO:teuthology.orchestra.run.trial127.stdout:created 2026-02-20T11:38:49.275524+0000 2026-02-20T11:38:49.279 INFO:teuthology.orchestra.run.trial127.stdout:min_mon_release 20 (tentacle) 2026-02-20T11:38:49.279 INFO:teuthology.orchestra.run.trial127.stdout:election_strategy: 1 2026-02-20T11:38:49.279 INFO:teuthology.orchestra.run.trial127.stdout:0: [v2:10.20.193.127:3300/0,v1:10.20.193.127:6789/0] mon.a 2026-02-20T11:38:49.279 INFO:teuthology.orchestra.run.trial127.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-20T11:38:49.280 DEBUG:teuthology.orchestra.run.trial127:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T11:38:49.337 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f7dd0554-3a23-4321-9783-f9664d26aacc... 2026-02-20T11:38:49.339 DEBUG:teuthology.orchestra.run.trial127:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-20T11:38:49.417 INFO:teuthology.orchestra.run.trial127.stdout:[global] 2026-02-20T11:38:49.418 INFO:teuthology.orchestra.run.trial127.stdout: chdir = "" 2026-02-20T11:38:49.418 INFO:teuthology.orchestra.run.trial127.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-20T11:38:49.418 INFO:teuthology.orchestra.run.trial127.stdout: auth supported = cephx 2026-02-20T11:38:49.418 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.418 INFO:teuthology.orchestra.run.trial127.stdout: filestore xattr use omap = true 2026-02-20T11:38:49.418 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.418 INFO:teuthology.orchestra.run.trial127.stdout: mon clock drift allowed = 1.000 2026-02-20T11:38:49.418 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.418 INFO:teuthology.orchestra.run.trial127.stdout: osd crush chooseleaf type = 0 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: auth debug = true 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: ms die on old message = true 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: ms die on bug = true 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: mon pg warn max object skew = 0 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: # disable pg_autoscaler by default for new pools 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.419 INFO:teuthology.orchestra.run.trial127.stdout: osd pool default size = 2 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: mon osd allow primary affinity = true 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: mon osd allow pg remap = true 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: mon warn on legacy crush tunables = false 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: mon warn on crush straw calc version zero = false 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: mon warn on no sortbitwise = false 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: mon warn on osd down out interval zero = false 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: mon warn on too few osds = false 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: mon_allow_pool_size_one = true 2026-02-20T11:38:49.420 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: osd default data pool replay window = 5 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: mon allow pool delete = true 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: mon cluster log file level = debug 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: debug asserts on shutdown = true 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: mon health detail to clog = false 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: mon host = 10.20.193.127 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-20T11:38:49.421 INFO:teuthology.orchestra.run.trial127.stdout: ms client mode = crc 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout: ms cluster mode = crc 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout: ms mon client mode = crc 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout: ms mon cluster mode = crc 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout: ms mon service mode = crc 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout: ms service mode = crc 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout: fsid = f7dd0554-3a23-4321-9783-f9664d26aacc 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout:[osd] 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout: osd journal size = 100 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout: osd scrub load threshold = 5.0 2026-02-20T11:38:49.422 INFO:teuthology.orchestra.run.trial127.stdout: osd scrub max interval = 600 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: osd mclock profile = high_recovery_ops 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: osd mclock skip benchmark = true 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: osd recover clone overlap = true 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: osd recovery max chunk = 1048576 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: osd debug shutdown = true 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: osd debug op order = true 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: osd debug verify stray on activate = true 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: osd debug trim objects = true 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.423 INFO:teuthology.orchestra.run.trial127.stdout: osd open classes on start = true 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: osd debug pg log writeout = true 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: osd deep scrub update digest min age = 30 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: osd map max advance = 10 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: journal zero on create = true 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: filestore ondisk finisher threads = 3 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: filestore apply finisher threads = 3 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: bdev debug aio = true 2026-02-20T11:38:49.424 INFO:teuthology.orchestra.run.trial127.stdout: osd debug misdirected ops = true 2026-02-20T11:38:49.425 INFO:teuthology.orchestra.run.trial127.stdout: debug monc = 20 2026-02-20T11:38:49.425 INFO:teuthology.orchestra.run.trial127.stdout: crimson cpu num = 4 2026-02-20T11:38:49.425 INFO:teuthology.orchestra.run.trial127.stdout: crimson osd obc lru size = 10 2026-02-20T11:38:49.425 INFO:teuthology.orchestra.run.trial127.stdout: debug ms = 20 2026-02-20T11:38:49.425 INFO:teuthology.orchestra.run.trial127.stdout: debug osd = 20 2026-02-20T11:38:49.425 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore = 20 2026-02-20T11:38:49.425 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore backref = 20 2026-02-20T11:38:49.425 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore cache = 20 2026-02-20T11:38:49.425 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore cleaner = 20 2026-02-20T11:38:49.425 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore device = 20 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore epm = 20 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore fixedkv tree = 20 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore journal = 20 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore lba = 20 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore odata = 20 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore omap = 20 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore onode = 20 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore t = 20 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: debug seastore tm = 20 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: osd objectstore = seastore 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: osd shutdown pgref assert = True 2026-02-20T11:38:49.426 INFO:teuthology.orchestra.run.trial127.stdout: seastore cachepin size pershard = 64M 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout: seastore cachepin type = 2Q 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout: seastore max concurrent transactions = 8 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout:[mgr] 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout: debug ms = 1 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout: debug mgr = 20 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout: debug mon = 20 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout: debug auth = 20 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout: mon reweight min pgs per osd = 4 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout: mon reweight min bytes per osd = 10 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout: mgr/telemetry/nag = false 2026-02-20T11:38:49.427 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout:[mon] 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: debug ms = 1 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: debug mon = 20 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: debug paxos = 20 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: debug auth = 20 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: mon data avail warn = 5 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: mon mgr mkfs grace = 240 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: mon reweight min pgs per osd = 4 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: mon osd reporter subtree level = osd 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: mon osd prime pg temp = true 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: mon reweight min bytes per osd = 10 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.428 INFO:teuthology.orchestra.run.trial127.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: auth mon ticket ttl = 660 # 11m 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: auth service ticket ttl = 240 # 4m 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: # don't complain about insecure global_id in the test suite 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: # 1m isn't quite enough 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: mon_down_mkfs_grace = 2m 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: mon_warn_on_filestore_osds = false 2026-02-20T11:38:49.429 INFO:teuthology.orchestra.run.trial127.stdout: mon min osdmap epochs = 50 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout: mon osdmap full prune interval = 2 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout: mon osdmap full prune min = 15 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout: mon osdmap full prune txsize = 2 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout: paxos service trim min = 10 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout: osd pool default crimson = True 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout:[client] 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout: rgw cache enabled = true 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout: rgw enable ops log = true 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout: rgw enable usage log = true 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-20T11:38:49.430 INFO:teuthology.orchestra.run.trial127.stdout:[mon.a] 2026-02-20T11:38:49.432 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-20T11:38:49.432 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-20T11:38:49.523 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-20T11:38:49.524 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:38:49.524 DEBUG:teuthology.orchestra.run.trial127:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-20T11:38:49.541 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:38:49.542 DEBUG:teuthology.orchestra.run.trial127:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-20T11:38:49.597 INFO:tasks.ceph:Sending monmap to node ubuntu@trial127.front.sepia.ceph.com 2026-02-20T11:38:49.597 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:38:49.597 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-20T11:38:49.597 DEBUG:teuthology.orchestra.run.trial127:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T11:38:49.671 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:38:49.671 DEBUG:teuthology.orchestra.run.trial127:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:38:49.729 INFO:tasks.ceph:Setting up mon nodes... 2026-02-20T11:38:49.729 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-20T11:38:49.730 DEBUG:teuthology.orchestra.run.trial127:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T11:38:49.820 INFO:teuthology.orchestra.run.trial127.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T11:38:49.822 INFO:tasks.ceph:Setting up mds nodes... 2026-02-20T11:38:49.823 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-20T11:38:49.823 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-20T11:38:49.864 INFO:teuthology.orchestra.run.trial127.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-20T11:38:49.876 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-20T11:38:49.876 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial127.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T11:38:49.876 DEBUG:teuthology.orchestra.run.trial127:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-20T11:38:49.942 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:38:49.943 INFO:tasks.ceph:role: osd.0 2026-02-20T11:38:49.943 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial127.front.sepia.ceph.com 2026-02-20T11:38:49.943 DEBUG:teuthology.orchestra.run.trial127:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-20T11:38:50.007 INFO:teuthology.orchestra.run.trial127.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:38:50.008 INFO:teuthology.orchestra.run.trial127.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:38:50.008 INFO:teuthology.orchestra.run.trial127.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:38:50.008 INFO:teuthology.orchestra.run.trial127.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:38:50.008 INFO:teuthology.orchestra.run.trial127.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:38:50.008 INFO:teuthology.orchestra.run.trial127.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:38:50.008 INFO:teuthology.orchestra.run.trial127.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:38:50.008 INFO:teuthology.orchestra.run.trial127.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:38:50.008 INFO:teuthology.orchestra.run.trial127.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:38:50.008 INFO:teuthology.orchestra.run.trial127.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:38:50.586 INFO:teuthology.orchestra.run.trial127.stdout:Discarding blocks...Done. 2026-02-20T11:38:50.588 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial127.front.sepia.ceph.com -o noatime 2026-02-20T11:38:50.588 DEBUG:teuthology.orchestra.run.trial127:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-20T11:38:50.658 DEBUG:teuthology.orchestra.run.trial127:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-20T11:38:50.726 DEBUG:teuthology.orchestra.run.trial127:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-20T11:38:50.790 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:38:50.790 INFO:tasks.ceph:role: osd.1 2026-02-20T11:38:50.790 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial127.front.sepia.ceph.com 2026-02-20T11:38:50.790 DEBUG:teuthology.orchestra.run.trial127:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-20T11:38:50.855 INFO:teuthology.orchestra.run.trial127.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:38:50.855 INFO:teuthology.orchestra.run.trial127.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:38:50.855 INFO:teuthology.orchestra.run.trial127.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:38:50.855 INFO:teuthology.orchestra.run.trial127.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:38:50.855 INFO:teuthology.orchestra.run.trial127.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:38:50.855 INFO:teuthology.orchestra.run.trial127.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:38:50.855 INFO:teuthology.orchestra.run.trial127.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:38:50.855 INFO:teuthology.orchestra.run.trial127.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:38:50.855 INFO:teuthology.orchestra.run.trial127.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:38:50.855 INFO:teuthology.orchestra.run.trial127.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:38:51.424 INFO:teuthology.orchestra.run.trial127.stdout:Discarding blocks...Done. 2026-02-20T11:38:51.426 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial127.front.sepia.ceph.com -o noatime 2026-02-20T11:38:51.426 DEBUG:teuthology.orchestra.run.trial127:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-20T11:38:51.499 DEBUG:teuthology.orchestra.run.trial127:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-20T11:38:51.565 DEBUG:teuthology.orchestra.run.trial127:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-20T11:38:51.631 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:38:51.631 INFO:tasks.ceph:role: osd.2 2026-02-20T11:38:51.631 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial127.front.sepia.ceph.com 2026-02-20T11:38:51.631 DEBUG:teuthology.orchestra.run.trial127:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-20T11:38:51.696 INFO:teuthology.orchestra.run.trial127.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:38:51.696 INFO:teuthology.orchestra.run.trial127.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:38:51.696 INFO:teuthology.orchestra.run.trial127.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:38:51.696 INFO:teuthology.orchestra.run.trial127.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:38:51.696 INFO:teuthology.orchestra.run.trial127.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:38:51.696 INFO:teuthology.orchestra.run.trial127.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:38:51.696 INFO:teuthology.orchestra.run.trial127.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:38:51.696 INFO:teuthology.orchestra.run.trial127.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:38:51.696 INFO:teuthology.orchestra.run.trial127.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:38:51.696 INFO:teuthology.orchestra.run.trial127.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:38:52.271 INFO:teuthology.orchestra.run.trial127.stdout:Discarding blocks...Done. 2026-02-20T11:38:52.273 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial127.front.sepia.ceph.com -o noatime 2026-02-20T11:38:52.274 DEBUG:teuthology.orchestra.run.trial127:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-20T11:38:52.343 DEBUG:teuthology.orchestra.run.trial127:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-20T11:38:52.410 DEBUG:teuthology.orchestra.run.trial127:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:38:52.564 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:38:52.615 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:52,614 seastar - Reactor backend: linux-aio 2026-02-20T11:38:52.625 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:52,623 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:38:52.625 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:52,623 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:38:52.627 INFO:teuthology.orchestra.run.trial127.stderr:==39759==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:38:52.682 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:52,681 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T11:38:52.682 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:52,681 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T11:38:52.682 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:52,681 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T11:38:52.683 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:52,681 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T11:38:52.683 INFO:teuthology.orchestra.run.trial127.stderr:DEBUG 2026-02-20 11:38:52,681 [shard 0:main] seastar - reactor::drain 2026-02-20T11:38:54.732 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:38:54.733 INFO:teuthology.orchestra.run.trial127.stderr:Suppressions used: 2026-02-20T11:38:54.733 INFO:teuthology.orchestra.run.trial127.stderr: count bytes template 2026-02-20T11:38:54.733 INFO:teuthology.orchestra.run.trial127.stderr: 1 8 MallocExtension::Register 2026-02-20T11:38:54.733 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:38:54.733 INFO:teuthology.orchestra.run.trial127.stderr: 2026-02-20T11:38:54.752 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:38:54.752 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:38:54.795 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:38:54.795 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:38:54.801 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:54,800 seastar - Reactor backend: linux-aio 2026-02-20T11:38:54.811 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:54,810 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:38:54.811 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:54,810 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:38:54.815 INFO:teuthology.orchestra.run.trial127.stderr:==39757==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:38:54.816 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:54,814 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:38:54.874 INFO:teuthology.orchestra.run.trial127.stderr:DEBUG 2026-02-20 11:38:54,873 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T11:38:54.956 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfa81f /lib64/libasan.so.8+0xfd191 /lib64/libasan.so.8+0xee859 0x9cf0b5b 0x9d70916 0x9d70a2c 0x9d70b08 0x9d711c4 0x9d70e3d 0x9d7132b 0x9d7172f 0x9d719ec 0x9f1a6cd 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:54.956 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c1b /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9cf0b5b 0x9ee0e7e 0x9ee0fac 0x9f137b3 0x9f13f56 0x9f1e423 0x78bcf19 0x7b9c081 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:54.956 INFO:teuthology.orchestra.run.trial127.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffaad27ef7 0xffffffffaa9f7f16 0xffffffffaa9fb8d0 0xffffffffaa9fcc31 0xffffffffaaa24893 0xffffffffaaa24a55 0xffffffffaa9cb80e 0xffffffffaa9d3aeb 0xffffffffaa9d3f4b 0xffffffffaa6910d2 0xffffffffab34d451 0xffffffffab400ba2 2026-02-20T11:38:54.957 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x157db 0x10a3723 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e1ecb8 0x9e1f27a 0x9e27721 0x9e27e01 0x9f17e54 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:54.988 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0xc28a67 0xc28f88 0xc84842 0xc849f9 0xc869b6 0xc86fcc 0xc8d3ef 0xc8d92e 0xa7b410a 0xc7ecae 0x7ac2aec 0x79c28e8 0x7ab6f00 0xa81afb8 0xa81b5aa 0x79c2cef 0x7a42fcc 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:38:55.031 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x12974a /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x9e1ea56 0x9e1ecb8 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7f67185 0x7fb45b4 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:55.031 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xf4e9a /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xfd19a /lib64/libasan.so.8+0xedcb5 0xe419a1 0x9edc84a 0x9edd05c 0x9eebabe 0x9eee549 0x9eee75a 0x9eee806 0x9f17355 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:55.032 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9e2ccb8 0x9efa76f 0x9efa937 0x9efaa98 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:55.055 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:38:55.097 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x297f494 0x29d89ca 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:55.103 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6583 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:55.104 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:55.239 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16f342 /lib64/libasan.so.8+0x4d695 0xe2fd77 0x7b9bfb2 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:55.239 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9e1ebd5 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:55.239 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x12985d /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x10a3723 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e20729 0x9e20c76 0x9e20e75 0x9f0d69d 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:55.322 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xf4e8f /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:38:55.337 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:55.338 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:55.338 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d8989 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:38:57.519 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:38:57.519 INFO:teuthology.orchestra.run.trial127.stderr:Suppressions used: 2026-02-20T11:38:57.519 INFO:teuthology.orchestra.run.trial127.stderr: count bytes template 2026-02-20T11:38:57.520 INFO:teuthology.orchestra.run.trial127.stderr: 1 8 MallocExtension::Register 2026-02-20T11:38:57.520 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:38:57.520 INFO:teuthology.orchestra.run.trial127.stderr: 2026-02-20T11:38:57.555 DEBUG:teuthology.orchestra.run.trial127:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-20T11:38:57.580 DEBUG:teuthology.orchestra.run.trial127:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:38:57.734 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:38:57.787 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:57,785 seastar - Reactor backend: linux-aio 2026-02-20T11:38:57.801 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:57,800 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:38:57.802 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:57,800 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:38:57.804 INFO:teuthology.orchestra.run.trial127.stderr:==39824==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:38:57.859 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:57,857 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T11:38:57.859 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:57,857 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T11:38:57.860 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:57,857 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T11:38:57.860 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:57,857 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T11:38:57.860 INFO:teuthology.orchestra.run.trial127.stderr:DEBUG 2026-02-20 11:38:57,858 [shard 0:main] seastar - reactor::drain 2026-02-20T11:38:59.914 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:38:59.914 INFO:teuthology.orchestra.run.trial127.stderr:Suppressions used: 2026-02-20T11:38:59.914 INFO:teuthology.orchestra.run.trial127.stderr: count bytes template 2026-02-20T11:38:59.915 INFO:teuthology.orchestra.run.trial127.stderr: 1 8 MallocExtension::Register 2026-02-20T11:38:59.915 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:38:59.915 INFO:teuthology.orchestra.run.trial127.stderr: 2026-02-20T11:38:59.934 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:38:59.934 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:38:59.979 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:38:59.979 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:38:59.985 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:59,984 seastar - Reactor backend: linux-aio 2026-02-20T11:38:59.994 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:59,993 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:38:59.994 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:59,993 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:38:59.998 INFO:teuthology.orchestra.run.trial127.stderr:==39822==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:38:59.998 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:38:59,996 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:39:00.056 INFO:teuthology.orchestra.run.trial127.stderr:DEBUG 2026-02-20 11:39:00,055 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T11:39:00.137 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9ddc7f6 0x9f06f75 0x9f07573 0x9f07a5e 0x9f07df1 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.138 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151e3 0x1138fc8 0xf4864f 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e20729 0x9e20c76 0x9e20e75 0x9f0d69d 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.138 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x30f33e2 0x6e083ec 0x9efa8e2 0x9efaa98 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.170 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x31fb04e 0x9e1e9f0 0x9e1ecb8 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7ba2449 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:39:00.213 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xea9f7 0xe416d4 0xe41a8f 0x9efa612 0x9efa7ce 0x9efa937 0x9efaa62 0x9efaf8c 0x9f19778 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7f67185 0x7fb45b4 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.213 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191cf5 /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x1367182 0x29df4ae 0x7f6c018 0x7f6c5ce 0x7f6c6fa 0x7f6f26f 0x7f6fec3 0x7f70184 0x7e5a8a9 0x7fb3f68 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.213 INFO:teuthology.orchestra.run.trial127.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffaac54dec 0xffffffffaaa0f59b 0xffffffffaa9cb858 0xffffffffaa9d3aeb 0xffffffffaa9d3f4b 0xffffffffaa6910d2 0xffffffffab34d451 0xffffffffab400ba2 2026-02-20T11:39:00.237 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:39:00.278 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d83ab 0x7d69b2a 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.278 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.279 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x12978a /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.404 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x30f33e2 0x6e083ec 0x9efa8e2 0x9efaa62 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.405 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x1138fc8 0xf4864f 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e20729 0x9f0b606 0x9f0d258 0x9f0d8b2 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.405 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c1b /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9cf0b5b 0x9d70916 0x9d70a2c 0x9d70b08 0x9d711c4 0x9d70e3d 0x9d7132b 0x9d7172f 0x9d719ec 0x9f1a6cd 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.405 INFO:teuthology.orchestra.run.trial127.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffaacc5c91 0xffffffffaa9f9163 0xffffffffaa9f9774 0xffffffffaa9fb709 0xffffffffaa9fcc31 0xffffffffaaa24893 0xffffffffaaa24a55 0xffffffffaa9cb80e 0xffffffffaa9d3aeb 0xffffffffaa9d3f4b 0xffffffffaa6910d2 0xffffffffab34d451 0xffffffffab400ba2 2026-02-20T11:39:00.487 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:39:00.503 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d83ab 0x7d69b2a 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.504 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6583 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:00.504 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x7d69ae7 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:02.672 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:39:02.673 INFO:teuthology.orchestra.run.trial127.stderr:Suppressions used: 2026-02-20T11:39:02.673 INFO:teuthology.orchestra.run.trial127.stderr: count bytes template 2026-02-20T11:39:02.673 INFO:teuthology.orchestra.run.trial127.stderr: 1 8 MallocExtension::Register 2026-02-20T11:39:02.673 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:39:02.673 INFO:teuthology.orchestra.run.trial127.stderr: 2026-02-20T11:39:02.708 DEBUG:teuthology.orchestra.run.trial127:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-20T11:39:02.733 DEBUG:teuthology.orchestra.run.trial127:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:39:02.889 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:39:02.942 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:39:02,940 seastar - Reactor backend: linux-aio 2026-02-20T11:39:02.948 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:39:02,947 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:39:02.949 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:39:02,947 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:39:02.951 INFO:teuthology.orchestra.run.trial127.stderr:==39889==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:39:03.007 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:39:03,004 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T11:39:03.008 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:39:03,004 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T11:39:03.008 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:39:03,004 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T11:39:03.008 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:39:03,004 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T11:39:03.008 INFO:teuthology.orchestra.run.trial127.stderr:DEBUG 2026-02-20 11:39:03,004 [shard 0:main] seastar - reactor::drain 2026-02-20T11:39:05.056 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:39:05.057 INFO:teuthology.orchestra.run.trial127.stderr:Suppressions used: 2026-02-20T11:39:05.057 INFO:teuthology.orchestra.run.trial127.stderr: count bytes template 2026-02-20T11:39:05.057 INFO:teuthology.orchestra.run.trial127.stderr: 1 8 MallocExtension::Register 2026-02-20T11:39:05.057 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:39:05.057 INFO:teuthology.orchestra.run.trial127.stderr: 2026-02-20T11:39:05.077 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:39:05.077 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:39:05.122 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:39:05.122 INFO:teuthology.orchestra.run.trial127.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:39:05.128 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:39:05,127 seastar - Reactor backend: linux-aio 2026-02-20T11:39:05.144 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:39:05,142 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:39:05.144 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:39:05,142 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:39:05.147 INFO:teuthology.orchestra.run.trial127.stderr:==39887==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:39:05.148 INFO:teuthology.orchestra.run.trial127.stderr:INFO 2026-02-20 11:39:05,146 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:39:05.206 INFO:teuthology.orchestra.run.trial127.stderr:DEBUG 2026-02-20 11:39:05,205 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T11:39:05.288 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9e1ebde 0x9e1f27a 0x9e27721 0x9e27e01 0x9f17e54 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.289 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0xcc3279 0x10a37e0 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e1ecb8 0x9e1f27a 0x9e27721 0x9e27e01 0x9f17e54 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.289 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b35d /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xb7b689 0x10a36ac 0xf4865a 0x9e1e492 0x9e1e78f 0x9e1ea02 0x9e20729 0x9f0b8b2 0x9f0d258 0x9f0d8b2 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.321 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xea80c 0xe416d4 0xe41a8f 0x9efa612 0x9efa7ce 0x9efa937 0x9efaa98 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b8bc1a 0x7ba0607 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:39:05.364 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x16c8b 0x9f0b988 0x9f0d258 0x9f0d8b2 0x9f0ddf8 0x9f19c81 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x911c2c7 0x911d4dc 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.364 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191ccd /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x1367182 0x29df4ae 0x7f6c018 0x7f6c5ce 0x7f6c6fa 0x7f6f26f 0x7f6fec3 0x7f70184 0x7e5a8a9 0x7fb3f68 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.365 INFO:teuthology.orchestra.run.trial127.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffaa9fcbb8 0xffffffffaaa24893 0xffffffffaaa24a55 0xffffffffaa9cb80e 0xffffffffaa9d3aeb 0xffffffffaa9d3f4b 0xffffffffaa6910d2 0xffffffffab34d451 0xffffffffab400ba2 2026-02-20T11:39:05.365 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0xc180e4 0xc8ce25 0xc8d522 0xc8d92e 0xa7b410a 0xc7ecae 0xa8183ce 0xa8188d8 0xa81b547 0x7f31616 0x7f326a5 0x79aad7b 0x7848ed0 0x7a41f12 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.389 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151ab 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:39:05.430 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.431 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.431 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d893e 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.560 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb13ef 0xb7b689 0x10a36ac 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e1ecb8 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.560 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc4d5 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9cf0b5b 0x9d70916 0x9d70a2c 0x9d70b08 0x9d70d84 0x9d7132b 0x9d7172f 0x9d719ec 0x9d72834 0x9ef0163 0x9ef025d 0x7b9c0be 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.560 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151dd 0x9dda46d 0x9f0b43a 0x9f0d258 0x9f0d8b2 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.643 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:39:05.659 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x7d69b08 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.659 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d89b8 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:05.660 INFO:teuthology.orchestra.run.trial127.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15277 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:39:07.844 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:39:07.845 INFO:teuthology.orchestra.run.trial127.stderr:Suppressions used: 2026-02-20T11:39:07.845 INFO:teuthology.orchestra.run.trial127.stderr: count bytes template 2026-02-20T11:39:07.845 INFO:teuthology.orchestra.run.trial127.stderr: 1 8 MallocExtension::Register 2026-02-20T11:39:07.845 INFO:teuthology.orchestra.run.trial127.stderr:----------------------------------------------------- 2026-02-20T11:39:07.845 INFO:teuthology.orchestra.run.trial127.stderr: 2026-02-20T11:39:07.884 DEBUG:teuthology.orchestra.run.trial127:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-20T11:39:07.909 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-20T11:39:07.909 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:39:07.909 DEBUG:teuthology.orchestra.run.trial127:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-20T11:39:07.973 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:39:07.973 DEBUG:teuthology.orchestra.run.trial127:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-20T11:39:08.038 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:39:08.038 DEBUG:teuthology.orchestra.run.trial127:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-20T11:39:08.102 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:39:08.102 DEBUG:teuthology.orchestra.run.trial127:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-20T11:39:08.168 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:39:08.168 DEBUG:teuthology.orchestra.run.trial127:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-20T11:39:08.226 INFO:tasks.ceph:Adding keys to all mons... 2026-02-20T11:39:08.226 DEBUG:teuthology.orchestra.run.trial127:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-20T11:39:08.289 INFO:teuthology.orchestra.run.trial127.stdout:[mgr.x] 2026-02-20T11:39:08.289 INFO:teuthology.orchestra.run.trial127.stdout: key = AQDJR5hpxXTOMBAAJP1ExS4jWs6DD11fcIRihQ== 2026-02-20T11:39:08.289 INFO:teuthology.orchestra.run.trial127.stdout:[osd.0] 2026-02-20T11:39:08.289 INFO:teuthology.orchestra.run.trial127.stdout: key = AQDOR5hpqlxLNBAAd9yTaGAHC1XEyKHE3cvk5w== 2026-02-20T11:39:08.289 INFO:teuthology.orchestra.run.trial127.stdout:[osd.1] 2026-02-20T11:39:08.290 INFO:teuthology.orchestra.run.trial127.stdout: key = AQDUR5hpia6DAxAAw7gGfo7Ste97Ned157It0g== 2026-02-20T11:39:08.290 INFO:teuthology.orchestra.run.trial127.stdout:[osd.2] 2026-02-20T11:39:08.290 INFO:teuthology.orchestra.run.trial127.stdout: key = AQDZR5hphAh6DBAAzDbzSxRl+RijvwQgEU2jFg== 2026-02-20T11:39:08.290 INFO:teuthology.orchestra.run.trial127.stdout:[client.0] 2026-02-20T11:39:08.290 INFO:teuthology.orchestra.run.trial127.stdout: key = AQDJR5hps5dzMxAA7VomyYI6z0fx7fwS9IMJww== 2026-02-20T11:39:08.291 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-20T11:39:08.379 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:39:08.427 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:39:08.474 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:39:08.523 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-20T11:39:08.574 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-20T11:39:08.574 DEBUG:teuthology.orchestra.run.trial127:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-20T11:39:08.599 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-20T11:39:08.686 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:08.684+0000 7fa317e32c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:08.686 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:08.684+0000 7fa317e32c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:08.697 DEBUG:teuthology.orchestra.run.trial127:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-20T11:39:08.721 DEBUG:teuthology.orchestra.run.trial127:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:39:08.777 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-20T11:39:08.778 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-20T11:39:08.778 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-20T11:39:08.819 INFO:tasks.ceph.mon.a:Started 2026-02-20T11:39:08.820 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-20T11:39:08.820 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-20T11:39:08.820 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-20T11:39:08.823 INFO:tasks.ceph.mgr.x:Started 2026-02-20T11:39:08.823 DEBUG:tasks.ceph:set 0 configs 2026-02-20T11:39:08.823 DEBUG:teuthology.orchestra.run.trial127:> sudo ceph --cluster ceph config dump 2026-02-20T11:39:08.883 INFO:tasks.ceph.mon.a.trial127.stderr:2026-02-20T11:39:08.881+0000 7f7851e49c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:08.884 INFO:tasks.ceph.mon.a.trial127.stderr:2026-02-20T11:39:08.881+0000 7f7851e49c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:08.884 INFO:tasks.ceph.mon.a.trial127.stderr:2026-02-20T11:39:08.881+0000 7f7851e49c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:08.885 INFO:tasks.ceph.mgr.x.trial127.stderr:2026-02-20T11:39:08.883+0000 7fe683e20fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:08.900 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:08.896+0000 7f9374ec2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:08.903 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:08.901+0000 7f9374ec2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:08.936 INFO:teuthology.orchestra.run.trial127.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-20T11:39:08.945 INFO:tasks.ceph:Setting crush tunables to default 2026-02-20T11:39:08.945 DEBUG:teuthology.orchestra.run.trial127:> sudo ceph --cluster ceph osd crush tunables default 2026-02-20T11:39:09.020 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:09.014+0000 7fad1065a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:09.022 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:09.020+0000 7fad1065a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:09.058 INFO:teuthology.orchestra.run.trial127.stderr:adjusted tunables profile to default 2026-02-20T11:39:09.067 INFO:tasks.ceph:check_enable_crimson: True 2026-02-20T11:39:09.067 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-20T11:39:09.067 DEBUG:teuthology.orchestra.run.trial127:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-20T11:39:09.089 INFO:tasks.ceph.mgr.x.trial127.stderr:2026-02-20T11:39:09.087+0000 7fe683e20fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:09.089 INFO:tasks.ceph.mgr.x.trial127.stderr:2026-02-20T11:39:09.087+0000 7fe683e20fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:09.140 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:09.135+0000 7f26b6d74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:09.142 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:09.140+0000 7f26b6d74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:09.181 INFO:tasks.ceph.mon.a.trial127.stderr:2026-02-20T11:39:09.179+0000 7f78491f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-20T11:39:09.182 INFO:tasks.ceph.mon.a.trial127.stderr:Please be aware that this feature is experimental, untested, 2026-02-20T11:39:09.182 INFO:tasks.ceph.mon.a.trial127.stderr:unsupported, and may result in data corruption, data loss, 2026-02-20T11:39:09.182 INFO:tasks.ceph.mon.a.trial127.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-20T11:39:09.182 INFO:tasks.ceph.mon.a.trial127.stderr:feature with important data. 2026-02-20T11:39:09.182 INFO:tasks.ceph.mon.a.trial127.stderr: 2026-02-20T11:39:09.939 INFO:tasks.ceph.mgr.x.trial127.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-20T11:39:09.939 INFO:tasks.ceph.mgr.x.trial127.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-20T11:39:09.939 INFO:tasks.ceph.mgr.x.trial127.stderr: from numpy import show_config as show_numpy_config 2026-02-20T11:39:10.070 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-20T11:39:10.070 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:39:10.070 DEBUG:teuthology.orchestra.run.trial127:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-20T11:39:10.135 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:39:10.135 DEBUG:teuthology.orchestra.run.trial127:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-20T11:39:10.198 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:39:10.198 DEBUG:teuthology.orchestra.run.trial127:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-20T11:39:10.263 DEBUG:teuthology.orchestra.run.trial127:> sudo ceph --cluster ceph osd new 982f21a5-e2eb-4f51-b6cf-27356488c525 0 2026-02-20T11:39:10.378 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:10.373+0000 7fc323e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:10.380 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:10.378+0000 7fc323e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:10.418 INFO:teuthology.orchestra.run.trial127.stdout:0 2026-02-20T11:39:10.427 DEBUG:teuthology.orchestra.run.trial127:> sudo ceph --cluster ceph osd new 2ff1b0af-eb05-4858-b8ce-5d3581c2527f 1 2026-02-20T11:39:10.501 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:10.496+0000 7f99abc1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:10.503 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:10.501+0000 7f99abc1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:10.541 INFO:teuthology.orchestra.run.trial127.stdout:1 2026-02-20T11:39:10.549 DEBUG:teuthology.orchestra.run.trial127:> sudo ceph --cluster ceph osd new 1cc9a239-0369-4885-b55c-f3046a325f2b 2 2026-02-20T11:39:10.624 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:10.619+0000 7f2bc1b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:10.626 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:10.624+0000 7f2bc1b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:10.664 INFO:teuthology.orchestra.run.trial127.stdout:2 2026-02-20T11:39:10.673 DEBUG:teuthology.orchestra.run.trial127:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-20T11:39:10.700 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-20T11:39:10.701 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-20T11:39:10.742 INFO:tasks.ceph.osd.0:Started 2026-02-20T11:39:10.742 DEBUG:teuthology.orchestra.run.trial127:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-20T11:39:10.772 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-20T11:39:10.772 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-20T11:39:10.814 INFO:tasks.ceph.osd.1:Started 2026-02-20T11:39:10.815 DEBUG:teuthology.orchestra.run.trial127:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-20T11:39:10.843 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-20T11:39:10.844 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-20T11:39:10.886 INFO:tasks.ceph.osd.2:Started 2026-02-20T11:39:10.887 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:39:10.968 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:10.962+0000 7f542cfdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:10.970 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:10.968+0000 7f542cfdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:11.003 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:39:11.003 INFO:teuthology.orchestra.run.trial127.stdout:{"epoch":6,"fsid":"f7dd0554-3a23-4321-9783-f9664d26aacc","created":"2026-02-20T11:39:08.893977+0000","modified":"2026-02-20T11:39:10.663202+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:39:10.663202+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"982f21a5-e2eb-4f51-b6cf-27356488c525","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2ff1b0af-eb05-4858-b8ce-5d3581c2527f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1cc9a239-0369-4885-b55c-f3046a325f2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:39:11.013 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-20T11:39:11.014 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-20T11:39:11.316 DEBUG:teuthology.orchestra.run.trial127:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:39:11.385 INFO:teuthology.misc.health.trial127.stderr:2026-02-20T11:39:11.380+0000 7fe2dbe8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:11.387 INFO:teuthology.misc.health.trial127.stderr:2026-02-20T11:39:11.385+0000 7fe2dbe8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:11.419 INFO:teuthology.misc.health.trial127.stdout: 2026-02-20T11:39:11.420 INFO:teuthology.misc.health.trial127.stdout:{"epoch":6,"fsid":"f7dd0554-3a23-4321-9783-f9664d26aacc","created":"2026-02-20T11:39:08.893977+0000","modified":"2026-02-20T11:39:10.663202+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:39:10.663202+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"982f21a5-e2eb-4f51-b6cf-27356488c525","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2ff1b0af-eb05-4858-b8ce-5d3581c2527f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1cc9a239-0369-4885-b55c-f3046a325f2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:39:11.427 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T11:39:12.363 INFO:tasks.ceph.mgr.x.trial127.stderr:2026-02-20T11:39:12.360+0000 7fe59edbc640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-20T11:39:17.730 DEBUG:teuthology.orchestra.run.trial127:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:39:17.802 INFO:teuthology.misc.health.trial127.stderr:2026-02-20T11:39:17.797+0000 7efe6a99c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:17.805 INFO:teuthology.misc.health.trial127.stderr:2026-02-20T11:39:17.802+0000 7efe6a99c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:17.930 INFO:teuthology.misc.health.trial127.stdout: 2026-02-20T11:39:17.930 INFO:teuthology.misc.health.trial127.stdout:{"epoch":8,"fsid":"f7dd0554-3a23-4321-9783-f9664d26aacc","created":"2026-02-20T11:39:08.893977+0000","modified":"2026-02-20T11:39:17.390848+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:39:10.663202+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"982f21a5-e2eb-4f51-b6cf-27356488c525","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2ff1b0af-eb05-4858-b8ce-5d3581c2527f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1cc9a239-0369-4885-b55c-f3046a325f2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:39:17.943 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T11:39:24.245 DEBUG:teuthology.orchestra.run.trial127:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:39:24.316 INFO:teuthology.misc.health.trial127.stderr:2026-02-20T11:39:24.311+0000 7f7a4dacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:24.318 INFO:teuthology.misc.health.trial127.stderr:2026-02-20T11:39:24.315+0000 7f7a4dacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:24.430 INFO:teuthology.misc.health.trial127.stdout: 2026-02-20T11:39:24.430 INFO:teuthology.misc.health.trial127.stdout:{"epoch":14,"fsid":"f7dd0554-3a23-4321-9783-f9664d26aacc","created":"2026-02-20T11:39:08.893977+0000","modified":"2026-02-20T11:39:23.415759+0000","last_up_change":"2026-02-20T11:39:20.403763+0000","last_in_change":"2026-02-20T11:39:10.663202+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:39:21.373294+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"982f21a5-e2eb-4f51-b6cf-27356488c525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6803","nonce":928988629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6804","nonce":928988629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6812","nonce":928988629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6811","nonce":928988629}]},"public_addr":"10.20.193.127:6803/928988629","cluster_addr":"10.20.193.127:6804/928988629","heartbeat_back_addr":"10.20.193.127:6812/928988629","heartbeat_front_addr":"10.20.193.127:6811/928988629","state":["exists","up"]},{"osd":1,"uuid":"2ff1b0af-eb05-4858-b8ce-5d3581c2527f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6807","nonce":2643657845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6808","nonce":2643657845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6806","nonce":2643657845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6813","nonce":2643657845}]},"public_addr":"10.20.193.127:6807/2643657845","cluster_addr":"10.20.193.127:6808/2643657845","heartbeat_back_addr":"10.20.193.127:6806/2643657845","heartbeat_front_addr":"10.20.193.127:6813/2643657845","state":["exists","up"]},{"osd":2,"uuid":"1cc9a239-0369-4885-b55c-f3046a325f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6810","nonce":3657503840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6809","nonce":3657503840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6805","nonce":3657503840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6802","nonce":3657503840}]},"public_addr":"10.20.193.127:6810/3657503840","cluster_addr":"10.20.193.127:6809/3657503840","heartbeat_back_addr":"10.20.193.127:6805/3657503840","heartbeat_front_addr":"10.20.193.127:6802/3657503840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:39:24.439 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-20T11:39:24.439 INFO:tasks.ceph:Creating RBD pool 2026-02-20T11:39:24.439 DEBUG:teuthology.orchestra.run.trial127:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-20T11:39:24.513 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:24.506+0000 7f7d8699c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:24.515 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:24.512+0000 7f7d8699c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:25.422 INFO:teuthology.orchestra.run.trial127.stderr:pool 'rbd' created 2026-02-20T11:39:25.435 DEBUG:teuthology.orchestra.run.trial127:> rbd --cluster ceph pool init rbd 2026-02-20T11:39:25.476 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:25.472+0000 7f3831ee3240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:25.476 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:25.473+0000 7f3831ee3240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:25.479 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:25.476+0000 7f3831ee3240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:28.458 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-20T11:39:28.459 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-20T11:39:28.460 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-20T11:39:28.576 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:28.572+0000 7f40f905a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:28.579 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:28.576+0000 7f40f905a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:28.690 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:39:28.700 INFO:teuthology.orchestra.run.trial127.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-20T11:39:28.700 INFO:tasks.ceph_manager:config epoch is 1 2026-02-20T11:39:28.700 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-20T11:39:28.700 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-20T11:39:28.701 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-20T11:39:28.778 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:28.774+0000 7f1a4285a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:28.781 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:28.779+0000 7f1a4285a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:28.925 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:39:28.933 INFO:teuthology.orchestra.run.trial127.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6800","nonce":629351172},{"type":"v1","addr":"10.20.193.127:6801","nonce":629351172}]},"active_addr":"10.20.193.127:6801/629351172","active_change":"2026-02-20T11:39:11.355972+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.127:0","nonce":3400273667}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.127:0","nonce":476730465}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.127:0","nonce":2301111923}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.127:0","nonce":258461313}]}]} 2026-02-20T11:39:28.936 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-20T11:39:28.937 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-20T11:39:28.937 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:39:29.014 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.010+0000 7f4dbc3b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.016 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.014+0000 7f4dbc3b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.128 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:39:29.128 INFO:teuthology.orchestra.run.trial127.stdout:{"epoch":18,"fsid":"f7dd0554-3a23-4321-9783-f9664d26aacc","created":"2026-02-20T11:39:08.893977+0000","modified":"2026-02-20T11:39:28.426927+0000","last_up_change":"2026-02-20T11:39:20.403763+0000","last_in_change":"2026-02-20T11:39:10.663202+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:39:21.373294+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T11:39:24.632123+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"982f21a5-e2eb-4f51-b6cf-27356488c525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6803","nonce":928988629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6804","nonce":928988629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6812","nonce":928988629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6811","nonce":928988629}]},"public_addr":"10.20.193.127:6803/928988629","cluster_addr":"10.20.193.127:6804/928988629","heartbeat_back_addr":"10.20.193.127:6812/928988629","heartbeat_front_addr":"10.20.193.127:6811/928988629","state":["exists","up"]},{"osd":1,"uuid":"2ff1b0af-eb05-4858-b8ce-5d3581c2527f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6807","nonce":2643657845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6808","nonce":2643657845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6806","nonce":2643657845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6813","nonce":2643657845}]},"public_addr":"10.20.193.127:6807/2643657845","cluster_addr":"10.20.193.127:6808/2643657845","heartbeat_back_addr":"10.20.193.127:6806/2643657845","heartbeat_front_addr":"10.20.193.127:6813/2643657845","state":["exists","up"]},{"osd":2,"uuid":"1cc9a239-0369-4885-b55c-f3046a325f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6810","nonce":3657503840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6809","nonce":3657503840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6805","nonce":3657503840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6802","nonce":3657503840}]},"public_addr":"10.20.193.127:6810/3657503840","cluster_addr":"10.20.193.127:6809/3657503840","heartbeat_back_addr":"10.20.193.127:6805/3657503840","heartbeat_front_addr":"10.20.193.127:6802/3657503840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:39:29.138 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-20T11:39:29.138 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:39:29.217 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.213+0000 7f79b1f6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.220 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.218+0000 7f79b1f6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.332 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:39:29.332 INFO:teuthology.orchestra.run.trial127.stdout:{"epoch":18,"fsid":"f7dd0554-3a23-4321-9783-f9664d26aacc","created":"2026-02-20T11:39:08.893977+0000","modified":"2026-02-20T11:39:28.426927+0000","last_up_change":"2026-02-20T11:39:20.403763+0000","last_in_change":"2026-02-20T11:39:10.663202+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:39:21.373294+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T11:39:24.632123+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"982f21a5-e2eb-4f51-b6cf-27356488c525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6803","nonce":928988629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6804","nonce":928988629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6812","nonce":928988629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6811","nonce":928988629}]},"public_addr":"10.20.193.127:6803/928988629","cluster_addr":"10.20.193.127:6804/928988629","heartbeat_back_addr":"10.20.193.127:6812/928988629","heartbeat_front_addr":"10.20.193.127:6811/928988629","state":["exists","up"]},{"osd":1,"uuid":"2ff1b0af-eb05-4858-b8ce-5d3581c2527f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6807","nonce":2643657845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6808","nonce":2643657845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6806","nonce":2643657845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6813","nonce":2643657845}]},"public_addr":"10.20.193.127:6807/2643657845","cluster_addr":"10.20.193.127:6808/2643657845","heartbeat_back_addr":"10.20.193.127:6806/2643657845","heartbeat_front_addr":"10.20.193.127:6813/2643657845","state":["exists","up"]},{"osd":2,"uuid":"1cc9a239-0369-4885-b55c-f3046a325f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6810","nonce":3657503840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6809","nonce":3657503840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6805","nonce":3657503840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6802","nonce":3657503840}]},"public_addr":"10.20.193.127:6810/3657503840","cluster_addr":"10.20.193.127:6809/3657503840","heartbeat_back_addr":"10.20.193.127:6805/3657503840","heartbeat_front_addr":"10.20.193.127:6802/3657503840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:39:29.343 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-20T11:39:29.343 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-20T11:39:29.344 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-20T11:39:29.426 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.421+0000 7fb195a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.428 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.423+0000 7fc28ffa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.429 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.423+0000 7f3813a1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.429 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.426+0000 7fb195a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.430 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.428+0000 7fc28ffa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.431 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.429+0000 7f3813a1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.448 INFO:teuthology.orchestra.run.trial127.stdout:42949672970 2026-02-20T11:39:29.448 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T11:39:29.450 INFO:teuthology.orchestra.run.trial127.stdout:47244640265 2026-02-20T11:39:29.450 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T11:39:29.451 INFO:teuthology.orchestra.run.trial127.stdout:38654705675 2026-02-20T11:39:29.451 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T11:39:29.569 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.564+0000 7fe1feb0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.569 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.564+0000 7fefd5451640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.570 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.564+0000 7f321061a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.571 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.569+0000 7fefd5451640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.572 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.569+0000 7fe1feb0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.572 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:29.569+0000 7f321061a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:29.685 INFO:teuthology.orchestra.run.trial127.stdout:38654705673 2026-02-20T11:39:29.688 INFO:teuthology.orchestra.run.trial127.stdout:42949672970 2026-02-20T11:39:29.689 INFO:teuthology.orchestra.run.trial127.stdout:47244640261 2026-02-20T11:39:29.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705675 got 38654705673 for osd.2 2026-02-20T11:39:29.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672970 got 42949672970 for osd.0 2026-02-20T11:39:29.700 DEBUG:teuthology.parallel:result is None 2026-02-20T11:39:29.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640265 got 47244640261 for osd.1 2026-02-20T11:39:30.698 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T11:39:30.702 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T11:39:30.779 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:30.774+0000 7f33cf61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:30.782 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:30.779+0000 7f33cf61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:30.819 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:30.815+0000 7f05d821a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:30.822 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:30.820+0000 7f05d821a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:30.895 INFO:teuthology.orchestra.run.trial127.stdout:38654705673 2026-02-20T11:39:30.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705675 got 38654705673 for osd.2 2026-02-20T11:39:30.937 INFO:teuthology.orchestra.run.trial127.stdout:47244640261 2026-02-20T11:39:30.948 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640265 got 47244640261 for osd.1 2026-02-20T11:39:31.907 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T11:39:31.949 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T11:39:31.985 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:31.980+0000 7f0f38811640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:31.988 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:31.986+0000 7f0f38811640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:32.026 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:32.021+0000 7f2ca4a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:32.028 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:32.026+0000 7f2ca4a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:32.102 INFO:teuthology.orchestra.run.trial127.stdout:38654705675 2026-02-20T11:39:32.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705675 got 38654705675 for osd.2 2026-02-20T11:39:32.114 DEBUG:teuthology.parallel:result is None 2026-02-20T11:39:32.143 INFO:teuthology.orchestra.run.trial127.stdout:47244640265 2026-02-20T11:39:32.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640265 got 47244640265 for osd.1 2026-02-20T11:39:32.155 DEBUG:teuthology.parallel:result is None 2026-02-20T11:39:32.155 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-20T11:39:32.155 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:39:32.271 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:32.267+0000 7faa7e65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:32.273 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:32.271+0000 7faa7e65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:32.386 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:39:32.386 INFO:teuthology.orchestra.run.trial127.stderr:dumped all 2026-02-20T11:39:32.396 INFO:teuthology.orchestra.run.trial127.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2026-02-20T11:39:31.360499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2884,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497084,"statfs":{"total":160255967232,"available":160253014016,"internally_reserved":0,"allocated":2953216,"data_stored":2953216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.936315"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.441816+0000","last_change":"2026-02-20T11:39:28.445207+0000","last_active":"2026-02-20T11:39:28.441816+0000","last_peered":"2026-02-20T11:39:28.441816+0000","last_clean":"2026-02-20T11:39:28.441816+0000","last_became_active":"2026-02-20T11:39:26.458667+0000","last_became_peered":"2026-02-20T11:39:26.458667+0000","last_unstale":"2026-02-20T11:39:28.441816+0000","last_undegraded":"2026-02-20T11:39:28.441816+0000","last_fullsized":"2026-02-20T11:39:28.441816+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.443285+0000","last_change":"2026-02-20T11:39:28.445864+0000","last_active":"2026-02-20T11:39:28.443285+0000","last_peered":"2026-02-20T11:39:28.443285+0000","last_clean":"2026-02-20T11:39:28.443285+0000","last_became_active":"2026-02-20T11:39:26.468015+0000","last_became_peered":"2026-02-20T11:39:26.468015+0000","last_unstale":"2026-02-20T11:39:28.443285+0000","last_undegraded":"2026-02-20T11:39:28.443285+0000","last_fullsized":"2026-02-20T11:39:28.443285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.443373+0000","last_change":"2026-02-20T11:39:28.448158+0000","last_active":"2026-02-20T11:39:28.443373+0000","last_peered":"2026-02-20T11:39:28.443373+0000","last_clean":"2026-02-20T11:39:28.443373+0000","last_became_active":"2026-02-20T11:39:26.478805+0000","last_became_peered":"2026-02-20T11:39:26.478805+0000","last_unstale":"2026-02-20T11:39:28.443373+0000","last_undegraded":"2026-02-20T11:39:28.443373+0000","last_fullsized":"2026-02-20T11:39:28.443373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.443551+0000","last_change":"2026-02-20T11:39:28.445859+0000","last_active":"2026-02-20T11:39:28.443551+0000","last_peered":"2026-02-20T11:39:28.443551+0000","last_clean":"2026-02-20T11:39:28.443551+0000","last_became_active":"2026-02-20T11:39:26.472032+0000","last_became_peered":"2026-02-20T11:39:26.472032+0000","last_unstale":"2026-02-20T11:39:28.443551+0000","last_undegraded":"2026-02-20T11:39:28.443551+0000","last_fullsized":"2026-02-20T11:39:28.443551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'1","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.444205+0000","last_change":"2026-02-20T11:39:28.444205+0000","last_active":"2026-02-20T11:39:28.444205+0000","last_peered":"2026-02-20T11:39:28.444205+0000","last_clean":"2026-02-20T11:39:28.444205+0000","last_became_active":"2026-02-20T11:39:26.472432+0000","last_became_peered":"2026-02-20T11:39:26.472432+0000","last_unstale":"2026-02-20T11:39:28.444205+0000","last_undegraded":"2026-02-20T11:39:28.444205+0000","last_fullsized":"2026-02-20T11:39:28.444205+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.441483+0000","last_change":"2026-02-20T11:39:28.443886+0000","last_active":"2026-02-20T11:39:28.441483+0000","last_peered":"2026-02-20T11:39:28.441483+0000","last_clean":"2026-02-20T11:39:28.441483+0000","last_became_active":"2026-02-20T11:39:26.472666+0000","last_became_peered":"2026-02-20T11:39:26.472666+0000","last_unstale":"2026-02-20T11:39:28.441483+0000","last_undegraded":"2026-02-20T11:39:28.441483+0000","last_fullsized":"2026-02-20T11:39:28.441483+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.439457+0000","last_change":"2026-02-20T11:39:28.442460+0000","last_active":"2026-02-20T11:39:28.439457+0000","last_peered":"2026-02-20T11:39:28.439457+0000","last_clean":"2026-02-20T11:39:28.439457+0000","last_became_active":"2026-02-20T11:39:26.493646+0000","last_became_peered":"2026-02-20T11:39:26.493646+0000","last_unstale":"2026-02-20T11:39:28.439457+0000","last_undegraded":"2026-02-20T11:39:28.439457+0000","last_fullsized":"2026-02-20T11:39:28.439457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.444533+0000","last_change":"2026-02-20T11:39:28.448056+0000","last_active":"2026-02-20T11:39:28.444533+0000","last_peered":"2026-02-20T11:39:28.444533+0000","last_clean":"2026-02-20T11:39:28.444533+0000","last_became_active":"2026-02-20T11:39:26.463577+0000","last_became_peered":"2026-02-20T11:39:26.463577+0000","last_unstale":"2026-02-20T11:39:28.444533+0000","last_undegraded":"2026-02-20T11:39:28.444533+0000","last_fullsized":"2026-02-20T11:39:28.444533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":90,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.442174+0000","last_change":"2026-02-20T11:39:22.453510+0000","last_active":"2026-02-20T11:39:28.442174+0000","last_peered":"2026-02-20T11:39:28.442174+0000","last_clean":"2026-02-20T11:39:28.442174+0000","last_became_active":"2026-02-20T11:39:22.444155+0000","last_became_peered":"2026-02-20T11:39:22.444155+0000","last_unstale":"2026-02-20T11:39:28.442174+0000","last_undegraded":"2026-02-20T11:39:28.442174+0000","last_fullsized":"2026-02-20T11:39:28.442174+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_clean_scrub_stamp":"2026-02-20T11:39:21.408836+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253014016,"internally_reserved":0,"allocated":2953216,"data_stored":2953216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":0,"up_from":10,"seq":42949672970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1196,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165460,"statfs":{"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1316,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165340,"statfs":{"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":372,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166284,"statfs":{"total":53418655744,"available":53418274816,"internally_reserved":0,"allocated":380928,"data_stored":380928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418274816,"internally_reserved":0,"allocated":380928,"data_stored":380928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:39:32.398 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:39:32.475 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:32.471+0000 7f3f9d85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:32.477 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:32.475+0000 7f3f9d85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:32.588 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:39:32.588 INFO:teuthology.orchestra.run.trial127.stderr:dumped all 2026-02-20T11:39:32.598 INFO:teuthology.orchestra.run.trial127.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2026-02-20T11:39:31.360499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2884,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497084,"statfs":{"total":160255967232,"available":160253014016,"internally_reserved":0,"allocated":2953216,"data_stored":2953216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.936315"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.441816+0000","last_change":"2026-02-20T11:39:28.445207+0000","last_active":"2026-02-20T11:39:28.441816+0000","last_peered":"2026-02-20T11:39:28.441816+0000","last_clean":"2026-02-20T11:39:28.441816+0000","last_became_active":"2026-02-20T11:39:26.458667+0000","last_became_peered":"2026-02-20T11:39:26.458667+0000","last_unstale":"2026-02-20T11:39:28.441816+0000","last_undegraded":"2026-02-20T11:39:28.441816+0000","last_fullsized":"2026-02-20T11:39:28.441816+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.443285+0000","last_change":"2026-02-20T11:39:28.445864+0000","last_active":"2026-02-20T11:39:28.443285+0000","last_peered":"2026-02-20T11:39:28.443285+0000","last_clean":"2026-02-20T11:39:28.443285+0000","last_became_active":"2026-02-20T11:39:26.468015+0000","last_became_peered":"2026-02-20T11:39:26.468015+0000","last_unstale":"2026-02-20T11:39:28.443285+0000","last_undegraded":"2026-02-20T11:39:28.443285+0000","last_fullsized":"2026-02-20T11:39:28.443285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.443373+0000","last_change":"2026-02-20T11:39:28.448158+0000","last_active":"2026-02-20T11:39:28.443373+0000","last_peered":"2026-02-20T11:39:28.443373+0000","last_clean":"2026-02-20T11:39:28.443373+0000","last_became_active":"2026-02-20T11:39:26.478805+0000","last_became_peered":"2026-02-20T11:39:26.478805+0000","last_unstale":"2026-02-20T11:39:28.443373+0000","last_undegraded":"2026-02-20T11:39:28.443373+0000","last_fullsized":"2026-02-20T11:39:28.443373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.443551+0000","last_change":"2026-02-20T11:39:28.445859+0000","last_active":"2026-02-20T11:39:28.443551+0000","last_peered":"2026-02-20T11:39:28.443551+0000","last_clean":"2026-02-20T11:39:28.443551+0000","last_became_active":"2026-02-20T11:39:26.472032+0000","last_became_peered":"2026-02-20T11:39:26.472032+0000","last_unstale":"2026-02-20T11:39:28.443551+0000","last_undegraded":"2026-02-20T11:39:28.443551+0000","last_fullsized":"2026-02-20T11:39:28.443551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'1","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.444205+0000","last_change":"2026-02-20T11:39:28.444205+0000","last_active":"2026-02-20T11:39:28.444205+0000","last_peered":"2026-02-20T11:39:28.444205+0000","last_clean":"2026-02-20T11:39:28.444205+0000","last_became_active":"2026-02-20T11:39:26.472432+0000","last_became_peered":"2026-02-20T11:39:26.472432+0000","last_unstale":"2026-02-20T11:39:28.444205+0000","last_undegraded":"2026-02-20T11:39:28.444205+0000","last_fullsized":"2026-02-20T11:39:28.444205+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.441483+0000","last_change":"2026-02-20T11:39:28.443886+0000","last_active":"2026-02-20T11:39:28.441483+0000","last_peered":"2026-02-20T11:39:28.441483+0000","last_clean":"2026-02-20T11:39:28.441483+0000","last_became_active":"2026-02-20T11:39:26.472666+0000","last_became_peered":"2026-02-20T11:39:26.472666+0000","last_unstale":"2026-02-20T11:39:28.441483+0000","last_undegraded":"2026-02-20T11:39:28.441483+0000","last_fullsized":"2026-02-20T11:39:28.441483+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.439457+0000","last_change":"2026-02-20T11:39:28.442460+0000","last_active":"2026-02-20T11:39:28.439457+0000","last_peered":"2026-02-20T11:39:28.439457+0000","last_clean":"2026-02-20T11:39:28.439457+0000","last_became_active":"2026-02-20T11:39:26.493646+0000","last_became_peered":"2026-02-20T11:39:26.493646+0000","last_unstale":"2026-02-20T11:39:28.439457+0000","last_undegraded":"2026-02-20T11:39:28.439457+0000","last_fullsized":"2026-02-20T11:39:28.439457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.444533+0000","last_change":"2026-02-20T11:39:28.448056+0000","last_active":"2026-02-20T11:39:28.444533+0000","last_peered":"2026-02-20T11:39:28.444533+0000","last_clean":"2026-02-20T11:39:28.444533+0000","last_became_active":"2026-02-20T11:39:26.463577+0000","last_became_peered":"2026-02-20T11:39:26.463577+0000","last_unstale":"2026-02-20T11:39:28.444533+0000","last_undegraded":"2026-02-20T11:39:28.444533+0000","last_fullsized":"2026-02-20T11:39:28.444533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":90,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:39:28.442174+0000","last_change":"2026-02-20T11:39:22.453510+0000","last_active":"2026-02-20T11:39:28.442174+0000","last_peered":"2026-02-20T11:39:28.442174+0000","last_clean":"2026-02-20T11:39:28.442174+0000","last_became_active":"2026-02-20T11:39:22.444155+0000","last_became_peered":"2026-02-20T11:39:22.444155+0000","last_unstale":"2026-02-20T11:39:28.442174+0000","last_undegraded":"2026-02-20T11:39:28.442174+0000","last_fullsized":"2026-02-20T11:39:28.442174+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_clean_scrub_stamp":"2026-02-20T11:39:21.408836+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253014016,"internally_reserved":0,"allocated":2953216,"data_stored":2953216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":0,"up_from":10,"seq":42949672970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1196,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165460,"statfs":{"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1316,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165340,"statfs":{"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":372,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166284,"statfs":{"total":53418655744,"available":53418274816,"internally_reserved":0,"allocated":380928,"data_stored":380928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418274816,"internally_reserved":0,"allocated":380928,"data_stored":380928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:39:32.599 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-20T11:39:32.599 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-20T11:39:32.599 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-20T11:39:32.600 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-20T11:39:32.675 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:32.672+0000 7fc22325a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:32.677 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:39:32.675+0000 7fc22325a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:39:32.816 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:39:32.817 INFO:teuthology.orchestra.run.trial127.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-20T11:39:32.827 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-20T11:39:32.827 INFO:teuthology.run_tasks:Running task workunit... 2026-02-20T11:39:32.853 INFO:tasks.workunit:Pulling workunits from ref 445d5c9ba18a4040e70e25303230c36c77773acb 2026-02-20T11:39:32.853 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-20T11:39:32.853 DEBUG:teuthology.orchestra.run.trial127:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-20T11:39:32.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:39:32.871 INFO:teuthology.orchestra.run.trial127.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-20T11:39:32.871 DEBUG:teuthology.orchestra.run.trial127:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-20T11:39:32.928 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-20T11:39:32.928 DEBUG:teuthology.orchestra.run.trial127:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-20T11:39:32.984 INFO:tasks.workunit:timeout=3h 2026-02-20T11:39:32.984 INFO:tasks.workunit:cleanup=True 2026-02-20T11:39:32.985 DEBUG:teuthology.orchestra.run.trial127:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 445d5c9ba18a4040e70e25303230c36c77773acb 2026-02-20T11:39:33.046 INFO:tasks.workunit.client.0.trial127.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-20T11:39:54.536 INFO:tasks.workunit.client.0.trial127.stderr:Note: switching to '445d5c9ba18a4040e70e25303230c36c77773acb'. 2026-02-20T11:39:54.537 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-20T11:39:54.537 INFO:tasks.workunit.client.0.trial127.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-20T11:39:54.537 INFO:tasks.workunit.client.0.trial127.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-20T11:39:54.537 INFO:tasks.workunit.client.0.trial127.stderr:state without impacting any branches by switching back to a branch. 2026-02-20T11:39:54.537 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-20T11:39:54.537 INFO:tasks.workunit.client.0.trial127.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-20T11:39:54.537 INFO:tasks.workunit.client.0.trial127.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-20T11:39:54.537 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-20T11:39:54.538 INFO:tasks.workunit.client.0.trial127.stderr: git switch -c 2026-02-20T11:39:54.538 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-20T11:39:54.538 INFO:tasks.workunit.client.0.trial127.stderr:Or undo this operation with: 2026-02-20T11:39:54.538 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-20T11:39:54.538 INFO:tasks.workunit.client.0.trial127.stderr: git switch - 2026-02-20T11:39:54.538 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-20T11:39:54.538 INFO:tasks.workunit.client.0.trial127.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-20T11:39:54.538 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-20T11:39:54.538 INFO:tasks.workunit.client.0.trial127.stderr:HEAD is now at 445d5c9ba18 Merge PR #66907 into main 2026-02-20T11:39:54.540 DEBUG:teuthology.orchestra.run.trial127:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-20T11:39:54.598 INFO:tasks.workunit.client.0.trial127.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-20T11:39:54.600 INFO:tasks.workunit.client.0.trial127.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-20T11:39:54.600 INFO:tasks.workunit.client.0.trial127.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-20T11:39:54.688 INFO:tasks.workunit.client.0.trial127.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-20T11:39:54.719 INFO:tasks.workunit.client.0.trial127.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-20T11:39:54.743 INFO:tasks.workunit.client.0.trial127.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-20T11:39:54.744 INFO:tasks.workunit.client.0.trial127.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-20T11:39:54.745 INFO:tasks.workunit.client.0.trial127.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-20T11:39:54.769 INFO:tasks.workunit.client.0.trial127.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-20T11:39:54.773 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-20T11:39:54.773 DEBUG:teuthology.orchestra.run.trial127:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-20T11:39:54.831 INFO:tasks.workunit:Running workunits matching cls/test_cls_rbd.sh on client.0... 2026-02-20T11:39:54.834 INFO:tasks.workunit:Running workunit cls/test_cls_rbd.sh... 2026-02-20T11:39:54.834 DEBUG:teuthology.orchestra.run.trial127:workunit test cls/test_cls_rbd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=445d5c9ba18a4040e70e25303230c36c77773acb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh 2026-02-20T11:39:54.906 INFO:tasks.workunit.client.0.trial127.stdout:Running main() from gmock_main.cc 2026-02-20T11:39:54.907 INFO:tasks.workunit.client.0.trial127.stdout:[==========] Running 71 tests from 1 test suite. 2026-02-20T11:39:54.907 INFO:tasks.workunit.client.0.trial127.stdout:[----------] Global test environment set-up. 2026-02-20T11:39:54.907 INFO:tasks.workunit.client.0.trial127.stdout:[----------] 71 tests from TestClsRbd 2026-02-20T11:39:56.462 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.get_all_features 2026-02-20T11:39:56.661 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.get_all_features (199 ms) 2026-02-20T11:39:56.661 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.copyup 2026-02-20T11:39:56.818 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.copyup (157 ms) 2026-02-20T11:39:56.819 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.sparse_copyup 2026-02-20T11:39:56.917 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.sparse_copyup (98 ms) 2026-02-20T11:39:56.917 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.get_and_set_id 2026-02-20T11:39:56.972 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.get_and_set_id (54 ms) 2026-02-20T11:39:56.972 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.add_remove_child 2026-02-20T11:39:57.066 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.add_remove_child (94 ms) 2026-02-20T11:39:57.066 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.directory_methods 2026-02-20T11:39:57.310 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.directory_methods (243 ms) 2026-02-20T11:39:57.310 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.create 2026-02-20T11:39:57.413 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.create (102 ms) 2026-02-20T11:39:57.413 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.get_features 2026-02-20T11:39:57.429 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.get_features (16 ms) 2026-02-20T11:39:57.430 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.get_object_prefix 2026-02-20T11:39:57.445 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.get_object_prefix (16 ms) 2026-02-20T11:39:57.446 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.get_create_timestamp 2026-02-20T11:39:57.459 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.get_create_timestamp (13 ms) 2026-02-20T11:39:57.460 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.get_access_timestamp 2026-02-20T11:39:57.488 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.get_access_timestamp (29 ms) 2026-02-20T11:39:57.489 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.get_modify_timestamp 2026-02-20T11:39:57.504 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.get_modify_timestamp (15 ms) 2026-02-20T11:39:57.504 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.get_data_pool 2026-02-20T11:39:57.535 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.get_data_pool (30 ms) 2026-02-20T11:39:57.535 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.get_size 2026-02-20T11:39:57.575 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.get_size (40 ms) 2026-02-20T11:39:57.575 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.set_size 2026-02-20T11:39:57.622 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.set_size (46 ms) 2026-02-20T11:39:57.622 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.protection_status 2026-02-20T11:39:57.783 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.protection_status (161 ms) 2026-02-20T11:39:57.783 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.snapshot_limits 2026-02-20T11:39:57.873 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.snapshot_limits (90 ms) 2026-02-20T11:39:57.873 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.parents_v1 2026-02-20T11:39:58.325 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.parents_v1 (452 ms) 2026-02-20T11:39:58.325 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.parents_v2 2026-02-20T11:39:58.637 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.parents_v2 (311 ms) 2026-02-20T11:39:58.637 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.snapshots 2026-02-20T11:39:58.868 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.snapshots (230 ms) 2026-02-20T11:39:58.868 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.snapid_race 2026-02-20T11:39:58.910 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.snapid_race (42 ms) 2026-02-20T11:39:58.910 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.stripingv2 2026-02-20T11:39:58.991 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.stripingv2 (80 ms) 2026-02-20T11:39:58.991 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.object_map_save 2026-02-20T11:39:59.004 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.object_map_save (13 ms) 2026-02-20T11:39:59.004 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.object_map_resize 2026-02-20T11:39:59.049 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.object_map_resize (44 ms) 2026-02-20T11:39:59.049 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.object_map_update 2026-02-20T11:39:59.092 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.object_map_update (42 ms) 2026-02-20T11:39:59.092 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.object_map_load_enoent 2026-02-20T11:39:59.094 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.object_map_load_enoent (2 ms) 2026-02-20T11:39:59.094 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.object_map_snap_add 2026-02-20T11:39:59.132 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.object_map_snap_add (37 ms) 2026-02-20T11:39:59.132 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.object_map_snap_remove 2026-02-20T11:39:59.169 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.object_map_snap_remove (37 ms) 2026-02-20T11:39:59.169 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.flags 2026-02-20T11:39:59.221 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.flags (51 ms) 2026-02-20T11:39:59.221 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.metadata 2026-02-20T11:39:59.562 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.metadata (340 ms) 2026-02-20T11:39:59.562 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.set_features 2026-02-20T11:39:59.623 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.set_features (61 ms) 2026-02-20T11:39:59.624 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.mirror 2026-02-20T11:39:59.964 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.mirror (340 ms) 2026-02-20T11:39:59.964 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.mirror_image 2026-02-20T11:40:00.119 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.mirror_image (155 ms) 2026-02-20T11:40:00.119 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.mirror_image_status 2026-02-20T11:40:30.256 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.mirror_image_status (30137 ms) 2026-02-20T11:40:30.256 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.mirror_image_map 2026-02-20T11:40:31.238 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.mirror_image_map (981 ms) 2026-02-20T11:40:31.238 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.mirror_instances 2026-02-20T11:40:31.304 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.mirror_instances (66 ms) 2026-02-20T11:40:31.305 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.mirror_snapshot 2026-02-20T11:40:31.428 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.mirror_snapshot (123 ms) 2026-02-20T11:40:31.428 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_dir_list 2026-02-20T11:40:31.453 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_dir_list (24 ms) 2026-02-20T11:40:31.453 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_dir_add 2026-02-20T11:40:31.474 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_dir_add (20 ms) 2026-02-20T11:40:31.474 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.dir_add_already_existing 2026-02-20T11:40:31.511 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.dir_add_already_existing (37 ms) 2026-02-20T11:40:31.511 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_dir_rename 2026-02-20T11:40:31.569 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_dir_rename (58 ms) 2026-02-20T11:40:31.570 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_dir_remove 2026-02-20T11:40:31.600 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_dir_remove (31 ms) 2026-02-20T11:40:31.601 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_dir_remove_missing 2026-02-20T11:40:31.639 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_dir_remove_missing (38 ms) 2026-02-20T11:40:31.639 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_image_add 2026-02-20T11:40:31.657 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_image_add (18 ms) 2026-02-20T11:40:31.657 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_image_remove 2026-02-20T11:40:31.687 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_image_remove (29 ms) 2026-02-20T11:40:31.687 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_image_list 2026-02-20T11:40:31.712 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_image_list (25 ms) 2026-02-20T11:40:31.713 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_image_clean 2026-02-20T11:40:31.747 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_image_clean (34 ms) 2026-02-20T11:40:31.747 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.image_group_add 2026-02-20T11:40:31.776 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.image_group_add (29 ms) 2026-02-20T11:40:31.776 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.image_group_remove 2026-02-20T11:40:31.811 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.image_group_remove (35 ms) 2026-02-20T11:40:31.812 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.image_group_get 2026-02-20T11:40:31.836 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.image_group_get (24 ms) 2026-02-20T11:40:31.836 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_snap_set_empty_name 2026-02-20T11:40:31.850 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_snap_set_empty_name (13 ms) 2026-02-20T11:40:31.850 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_snap_set_empty_id 2026-02-20T11:40:31.863 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_snap_set_empty_id (13 ms) 2026-02-20T11:40:31.864 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_snap_set_duplicate_id 2026-02-20T11:40:31.890 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_snap_set_duplicate_id (26 ms) 2026-02-20T11:40:31.890 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_snap_set_duplicate_name 2026-02-20T11:40:31.914 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_snap_set_duplicate_name (24 ms) 2026-02-20T11:40:31.914 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_snap_set 2026-02-20T11:40:31.934 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_snap_set (19 ms) 2026-02-20T11:40:31.934 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_snap_list 2026-02-20T11:40:32.006 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_snap_list (72 ms) 2026-02-20T11:40:32.006 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_snap_list_max_return 2026-02-20T11:40:32.207 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_snap_list_max_return (200 ms) 2026-02-20T11:40:32.207 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_snap_list_max_read 2026-02-20T11:40:34.163 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_snap_list_max_read (1956 ms) 2026-02-20T11:40:34.163 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_snap_remove 2026-02-20T11:40:34.193 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_snap_remove (30 ms) 2026-02-20T11:40:34.193 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_snap_remove_without_order 2026-02-20T11:40:34.232 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_snap_remove_without_order (38 ms) 2026-02-20T11:40:34.232 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.group_snap_get_by_id 2026-02-20T11:40:34.252 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.group_snap_get_by_id (20 ms) 2026-02-20T11:40:34.252 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.trash_methods 2026-02-20T11:40:34.318 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.trash_methods (65 ms) 2026-02-20T11:40:34.318 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.op_features 2026-02-20T11:40:34.380 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.op_features (62 ms) 2026-02-20T11:40:34.380 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.clone_parent 2026-02-20T11:40:34.609 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.clone_parent (228 ms) 2026-02-20T11:40:34.609 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.clone_parent_ns 2026-02-20T11:40:34.704 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.clone_parent_ns (94 ms) 2026-02-20T11:40:34.704 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.clone_child 2026-02-20T11:40:34.833 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.clone_child (129 ms) 2026-02-20T11:40:34.833 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.namespace_methods 2026-02-20T11:40:34.895 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.namespace_methods (61 ms) 2026-02-20T11:40:34.895 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.migration 2026-02-20T11:40:34.989 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.migration (94 ms) 2026-02-20T11:40:34.989 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.migration_v1 2026-02-20T11:40:35.070 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.migration_v1 (80 ms) 2026-02-20T11:40:35.070 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.assert_snapc_seq 2026-02-20T11:40:36.919 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.assert_snapc_seq (1848 ms) 2026-02-20T11:40:36.919 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsRbd.sparsify 2026-02-20T11:40:37.142 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsRbd.sparsify (223 ms) 2026-02-20T11:40:37.926 INFO:tasks.workunit.client.0.trial127.stdout:[----------] 71 tests from TestClsRbd (40680 ms total) 2026-02-20T11:40:37.926 INFO:tasks.workunit.client.0.trial127.stdout: 2026-02-20T11:40:37.926 INFO:tasks.workunit.client.0.trial127.stdout:[----------] Global test environment tear-down 2026-02-20T11:40:37.927 INFO:tasks.workunit.client.0.trial127.stdout:[==========] 71 tests from 1 test suite ran. (43019 ms total) 2026-02-20T11:40:37.927 INFO:tasks.workunit.client.0.trial127.stdout:[ PASSED ] 71 tests. 2026-02-20T11:40:37.930 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-20T11:40:37.930 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-20T11:40:37.998 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2026-02-20T11:40:38.001 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2026-02-20T11:40:38.002 DEBUG:teuthology.orchestra.run.trial127:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=445d5c9ba18a4040e70e25303230c36c77773acb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2026-02-20T11:40:38.074 INFO:tasks.workunit.client.0.trial127.stdout:Running main() from gmock_main.cc 2026-02-20T11:40:38.074 INFO:tasks.workunit.client.0.trial127.stdout:[==========] Running 11 tests from 1 test suite. 2026-02-20T11:40:38.074 INFO:tasks.workunit.client.0.trial127.stdout:[----------] Global test environment set-up. 2026-02-20T11:40:38.074 INFO:tasks.workunit.client.0.trial127.stdout:[----------] 11 tests from ClsLock 2026-02-20T11:40:38.074 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] ClsLock.TestMultiLocking 2026-02-20T11:40:40.236 INFO:tasks.workunit.client.0.trial127.stdout:lock: mylock 2026-02-20T11:40:40.237 INFO:tasks.workunit.client.0.trial127.stdout: lock_type: exclusive 2026-02-20T11:40:40.237 INFO:tasks.workunit.client.0.trial127.stdout: tag: 2026-02-20T11:40:40.237 INFO:tasks.workunit.client.0.trial127.stdout: lockers: 2026-02-20T11:40:40.237 INFO:tasks.workunit.client.0.trial127.stdout: client.4210 expiration=2026-02-20T11:42:40.187616+0000 addr=v1:10.20.193.127:0/1331333094 cookie= 2026-02-20T11:40:40.268 INFO:tasks.workunit.client.0.trial127.stdout:lock: mylock 2026-02-20T11:40:40.269 INFO:tasks.workunit.client.0.trial127.stdout: lock_type: shared 2026-02-20T11:40:40.269 INFO:tasks.workunit.client.0.trial127.stdout: tag: 2026-02-20T11:40:40.269 INFO:tasks.workunit.client.0.trial127.stdout: lockers: 2026-02-20T11:40:40.269 INFO:tasks.workunit.client.0.trial127.stdout: client.4210 expiration=2026-02-20T11:42:40.255802+0000 addr=v1:10.20.193.127:0/1331333094 cookie= 2026-02-20T11:40:40.269 INFO:tasks.workunit.client.0.trial127.stdout: client.4213 expiration=0.000000 addr=v1:10.20.193.127:0/3749062181 cookie= 2026-02-20T11:40:40.279 INFO:tasks.workunit.client.0.trial127.stdout:lock: mylock 2026-02-20T11:40:40.279 INFO:tasks.workunit.client.0.trial127.stdout: lock_type: shared 2026-02-20T11:40:40.279 INFO:tasks.workunit.client.0.trial127.stdout: tag: 2026-02-20T11:40:40.279 INFO:tasks.workunit.client.0.trial127.stdout: lockers: 2026-02-20T11:40:40.279 INFO:tasks.workunit.client.0.trial127.stdout: client.4213 expiration=0.000000 addr=v1:10.20.193.127:0/3749062181 cookie= 2026-02-20T11:40:40.937 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] ClsLock.TestMultiLocking (2862 ms) 2026-02-20T11:40:40.937 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] ClsLock.TestMeta 2026-02-20T11:40:43.200 INFO:tasks.workunit.client.0.trial127.stdout:lock: mylock 2026-02-20T11:40:43.200 INFO:tasks.workunit.client.0.trial127.stdout: lock_type: shared 2026-02-20T11:40:43.200 INFO:tasks.workunit.client.0.trial127.stdout: tag: 2026-02-20T11:40:43.201 INFO:tasks.workunit.client.0.trial127.stdout: lockers: 2026-02-20T11:40:43.201 INFO:tasks.workunit.client.0.trial127.stdout: client.4219 expiration=0.000000 addr=v1:10.20.193.127:0/330592448 cookie= 2026-02-20T11:40:43.219 INFO:tasks.workunit.client.0.trial127.stdout:lock: mylock 2026-02-20T11:40:43.220 INFO:tasks.workunit.client.0.trial127.stdout: lock_type: exclusive 2026-02-20T11:40:43.220 INFO:tasks.workunit.client.0.trial127.stdout: tag: new_tag 2026-02-20T11:40:43.220 INFO:tasks.workunit.client.0.trial127.stdout: lockers: 2026-02-20T11:40:43.220 INFO:tasks.workunit.client.0.trial127.stdout: client.4216 expiration=0.000000 addr=v1:10.20.193.127:0/4269837938 cookie= 2026-02-20T11:40:43.949 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] ClsLock.TestMeta (3012 ms) 2026-02-20T11:40:43.950 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] ClsLock.TestCookie 2026-02-20T11:40:46.173 INFO:tasks.workunit.client.0.trial127.stdout:lock: mylock 2026-02-20T11:40:46.173 INFO:tasks.workunit.client.0.trial127.stdout: lock_type: exclusive 2026-02-20T11:40:46.173 INFO:tasks.workunit.client.0.trial127.stdout: tag: 2026-02-20T11:40:46.174 INFO:tasks.workunit.client.0.trial127.stdout: lockers: 2026-02-20T11:40:46.203 INFO:tasks.workunit.client.0.trial127.stdout:lock: mylock 2026-02-20T11:40:46.203 INFO:tasks.workunit.client.0.trial127.stdout: lock_type: shared 2026-02-20T11:40:46.203 INFO:tasks.workunit.client.0.trial127.stdout: tag: 2026-02-20T11:40:46.203 INFO:tasks.workunit.client.0.trial127.stdout: lockers: 2026-02-20T11:40:46.203 INFO:tasks.workunit.client.0.trial127.stdout: client.4222 expiration=0.000000 addr=v1:10.20.193.127:0/1639418999 cookie= 2026-02-20T11:40:46.203 INFO:tasks.workunit.client.0.trial127.stdout: client.4222 expiration=0.000000 addr=v1:10.20.193.127:0/1639418999 cookie=new cookie 2026-02-20T11:40:46.962 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] ClsLock.TestCookie (3012 ms) 2026-02-20T11:40:46.962 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] ClsLock.TestMultipleLocks 2026-02-20T11:40:49.974 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] ClsLock.TestMultipleLocks (3011 ms) 2026-02-20T11:40:49.974 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] ClsLock.TestLockDuration 2026-02-20T11:40:57.995 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] ClsLock.TestLockDuration (8021 ms) 2026-02-20T11:40:57.996 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] ClsLock.TestAssertLocked 2026-02-20T11:41:01.005 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] ClsLock.TestAssertLocked (3009 ms) 2026-02-20T11:41:01.005 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] ClsLock.TestSetCookie 2026-02-20T11:41:04.021 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] ClsLock.TestSetCookie (3016 ms) 2026-02-20T11:41:04.021 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] ClsLock.TestRenew 2026-02-20T11:41:25.046 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] ClsLock.TestRenew (21025 ms) 2026-02-20T11:41:25.047 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2026-02-20T11:41:32.065 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (7018 ms) 2026-02-20T11:41:32.066 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2026-02-20T11:41:40.085 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (8019 ms) 2026-02-20T11:41:40.085 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2026-02-20T11:41:48.106 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (8021 ms) 2026-02-20T11:41:48.106 INFO:tasks.workunit.client.0.trial127.stdout:[----------] 11 tests from ClsLock (70032 ms total) 2026-02-20T11:41:48.106 INFO:tasks.workunit.client.0.trial127.stdout: 2026-02-20T11:41:48.106 INFO:tasks.workunit.client.0.trial127.stdout:[----------] Global test environment tear-down 2026-02-20T11:41:48.106 INFO:tasks.workunit.client.0.trial127.stdout:[==========] 11 tests from 1 test suite ran. (70032 ms total) 2026-02-20T11:41:48.106 INFO:tasks.workunit.client.0.trial127.stdout:[ PASSED ] 11 tests. 2026-02-20T11:41:48.109 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-20T11:41:48.109 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-20T11:41:48.182 INFO:tasks.workunit:Running workunits matching cls/test_cls_journal.sh on client.0... 2026-02-20T11:41:48.185 INFO:tasks.workunit:Running workunit cls/test_cls_journal.sh... 2026-02-20T11:41:48.185 DEBUG:teuthology.orchestra.run.trial127:workunit test cls/test_cls_journal.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=445d5c9ba18a4040e70e25303230c36c77773acb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_journal.sh 2026-02-20T11:41:48.261 INFO:tasks.workunit.client.0.trial127.stdout:Running main() from gmock_main.cc 2026-02-20T11:41:48.262 INFO:tasks.workunit.client.0.trial127.stdout:[==========] Running 27 tests from 1 test suite. 2026-02-20T11:41:48.262 INFO:tasks.workunit.client.0.trial127.stdout:[----------] Global test environment set-up. 2026-02-20T11:41:48.262 INFO:tasks.workunit.client.0.trial127.stdout:[----------] 27 tests from TestClsJournal 2026-02-20T11:41:50.115 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.Create 2026-02-20T11:41:50.322 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.Create (207 ms) 2026-02-20T11:41:50.323 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.MinimumSet 2026-02-20T11:41:50.362 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.MinimumSet (39 ms) 2026-02-20T11:41:50.362 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.MinimumSetStale 2026-02-20T11:41:50.410 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.MinimumSetStale (48 ms) 2026-02-20T11:41:50.410 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.MinimumSetOrderConstraint 2026-02-20T11:41:50.438 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.MinimumSetOrderConstraint (27 ms) 2026-02-20T11:41:50.438 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ActiveSet 2026-02-20T11:41:50.466 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ActiveSet (28 ms) 2026-02-20T11:41:50.466 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ActiveSetStale 2026-02-20T11:41:50.502 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ActiveSetStale (36 ms) 2026-02-20T11:41:50.502 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.CreateDuplicate 2026-02-20T11:41:50.521 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.CreateDuplicate (18 ms) 2026-02-20T11:41:50.521 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.GetClient 2026-02-20T11:41:50.548 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.GetClient (26 ms) 2026-02-20T11:41:50.548 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ClientRegister 2026-02-20T11:41:50.572 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ClientRegister (24 ms) 2026-02-20T11:41:50.573 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ClientRegisterDuplicate 2026-02-20T11:41:50.602 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ClientRegisterDuplicate (29 ms) 2026-02-20T11:41:50.602 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ClientUpdateData 2026-02-20T11:41:50.642 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ClientUpdateData (39 ms) 2026-02-20T11:41:50.642 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ClientUpdateState 2026-02-20T11:41:50.682 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ClientUpdateState (40 ms) 2026-02-20T11:41:50.682 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ClientUnregister 2026-02-20T11:41:50.713 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ClientUnregister (31 ms) 2026-02-20T11:41:50.714 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ClientUnregisterDNE 2026-02-20T11:41:50.751 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ClientUnregisterDNE (37 ms) 2026-02-20T11:41:50.751 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ClientUnregisterPruneTags 2026-02-20T11:41:52.015 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ClientUnregisterPruneTags (1264 ms) 2026-02-20T11:41:52.015 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ClientCommit 2026-02-20T11:41:52.048 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ClientCommit (32 ms) 2026-02-20T11:41:52.048 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ClientCommitInvalid 2026-02-20T11:41:52.077 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ClientCommitInvalid (29 ms) 2026-02-20T11:41:52.077 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ClientCommitDNE 2026-02-20T11:41:52.085 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ClientCommitDNE (8 ms) 2026-02-20T11:41:52.085 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.ClientList 2026-02-20T11:41:53.991 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.ClientList (1906 ms) 2026-02-20T11:41:53.992 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.GetNextTagTid 2026-02-20T11:41:54.031 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.GetNextTagTid (39 ms) 2026-02-20T11:41:54.031 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.TagCreate 2026-02-20T11:41:54.122 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.TagCreate (90 ms) 2026-02-20T11:41:54.122 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.TagCreatePrunesTags 2026-02-20T11:41:54.206 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.TagCreatePrunesTags (83 ms) 2026-02-20T11:41:54.206 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.TagList 2026-02-20T11:41:55.374 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.TagList (1167 ms) 2026-02-20T11:41:55.374 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.GuardAppend 2026-02-20T11:41:55.385 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.GuardAppend (11 ms) 2026-02-20T11:41:55.385 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.GuardAppendDNE 2026-02-20T11:41:55.388 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.GuardAppendDNE (2 ms) 2026-02-20T11:41:55.388 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.GuardAppendOverflow 2026-02-20T11:41:55.413 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.GuardAppendOverflow (25 ms) 2026-02-20T11:41:55.414 INFO:tasks.workunit.client.0.trial127.stdout:[ RUN ] TestClsJournal.Append 2026-02-20T11:41:55.477 INFO:tasks.workunit.client.0.trial127.stdout:padding is not enabled 2026-02-20T11:41:55.478 INFO:tasks.workunit.client.0.trial127.stdout:[ OK ] TestClsJournal.Append (63 ms) 2026-02-20T11:41:56.445 INFO:tasks.workunit.client.0.trial127.stdout:[----------] 27 tests from TestClsJournal (5361 ms total) 2026-02-20T11:41:56.446 INFO:tasks.workunit.client.0.trial127.stdout: 2026-02-20T11:41:56.446 INFO:tasks.workunit.client.0.trial127.stdout:[----------] Global test environment tear-down 2026-02-20T11:41:56.446 INFO:tasks.workunit.client.0.trial127.stdout:[==========] 27 tests from 1 test suite ran. (8183 ms total) 2026-02-20T11:41:56.446 INFO:tasks.workunit.client.0.trial127.stdout:[ PASSED ] 27 tests. 2026-02-20T11:41:56.448 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-20T11:41:56.448 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-20T11:41:56.515 INFO:tasks.workunit:Stopping ['cls/test_cls_rbd.sh', 'cls/test_cls_lock.sh', 'cls/test_cls_journal.sh'] on client.0... 2026-02-20T11:41:56.515 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-20T11:41:56.919 DEBUG:teuthology.parallel:result is None 2026-02-20T11:41:56.919 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-20T11:41:56.945 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-20T11:41:56.945 DEBUG:teuthology.orchestra.run.trial127:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-20T11:41:57.001 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-20T11:41:57.002 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-20T11:41:57.017 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-20T11:41:57.017 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:41:57.117 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:57.112+0000 7fd83c65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:57.119 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:57.117+0000 7fd83c65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:57.229 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:41:57.229 INFO:teuthology.orchestra.run.trial127.stderr:dumped all 2026-02-20T11:41:57.239 INFO:teuthology.orchestra.run.trial127.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2026-02-20T11:41:56.440581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":10284,"kb_used_data":10284,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156489684,"statfs":{"total":160255967232,"available":160245436416,"internally_reserved":0,"allocated":10530816,"data_stored":10530816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.056577"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126452+0000","last_change":"2026-02-20T11:39:28.445207+0000","last_active":"2026-02-20T11:41:51.126452+0000","last_peered":"2026-02-20T11:41:51.126452+0000","last_clean":"2026-02-20T11:41:51.126452+0000","last_became_active":"2026-02-20T11:39:26.458667+0000","last_became_peered":"2026-02-20T11:39:26.458667+0000","last_unstale":"2026-02-20T11:41:51.126452+0000","last_undegraded":"2026-02-20T11:41:51.126452+0000","last_fullsized":"2026-02-20T11:41:51.126452+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126320+0000","last_change":"2026-02-20T11:39:28.445864+0000","last_active":"2026-02-20T11:41:51.126320+0000","last_peered":"2026-02-20T11:41:51.126320+0000","last_clean":"2026-02-20T11:41:51.126320+0000","last_became_active":"2026-02-20T11:39:26.468015+0000","last_became_peered":"2026-02-20T11:39:26.468015+0000","last_unstale":"2026-02-20T11:41:51.126320+0000","last_undegraded":"2026-02-20T11:41:51.126320+0000","last_fullsized":"2026-02-20T11:41:51.126320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.128542+0000","last_change":"2026-02-20T11:39:28.448158+0000","last_active":"2026-02-20T11:41:51.128542+0000","last_peered":"2026-02-20T11:41:51.128542+0000","last_clean":"2026-02-20T11:41:51.128542+0000","last_became_active":"2026-02-20T11:39:26.478805+0000","last_became_peered":"2026-02-20T11:39:26.478805+0000","last_unstale":"2026-02-20T11:41:51.128542+0000","last_undegraded":"2026-02-20T11:41:51.128542+0000","last_fullsized":"2026-02-20T11:41:51.128542+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.127689+0000","last_change":"2026-02-20T11:39:28.445859+0000","last_active":"2026-02-20T11:41:51.127689+0000","last_peered":"2026-02-20T11:41:51.127689+0000","last_clean":"2026-02-20T11:41:51.127689+0000","last_became_active":"2026-02-20T11:39:26.472032+0000","last_became_peered":"2026-02-20T11:39:26.472032+0000","last_unstale":"2026-02-20T11:41:51.127689+0000","last_undegraded":"2026-02-20T11:41:51.127689+0000","last_fullsized":"2026-02-20T11:41:51.127689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":158,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126559+0000","last_change":"2026-02-20T11:39:28.444205+0000","last_active":"2026-02-20T11:41:51.126559+0000","last_peered":"2026-02-20T11:41:51.126559+0000","last_clean":"2026-02-20T11:41:51.126559+0000","last_became_active":"2026-02-20T11:39:26.472432+0000","last_became_peered":"2026-02-20T11:39:26.472432+0000","last_unstale":"2026-02-20T11:41:51.126559+0000","last_undegraded":"2026-02-20T11:41:51.126559+0000","last_fullsized":"2026-02-20T11:41:51.126559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.143152+0000","last_change":"2026-02-20T11:39:28.443886+0000","last_active":"2026-02-20T11:41:51.143152+0000","last_peered":"2026-02-20T11:41:51.143152+0000","last_clean":"2026-02-20T11:41:51.143152+0000","last_became_active":"2026-02-20T11:39:26.472666+0000","last_became_peered":"2026-02-20T11:39:26.472666+0000","last_unstale":"2026-02-20T11:41:51.143152+0000","last_undegraded":"2026-02-20T11:41:51.143152+0000","last_fullsized":"2026-02-20T11:41:51.143152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.140994+0000","last_change":"2026-02-20T11:39:28.442460+0000","last_active":"2026-02-20T11:41:51.140994+0000","last_peered":"2026-02-20T11:41:51.140994+0000","last_clean":"2026-02-20T11:41:51.140994+0000","last_became_active":"2026-02-20T11:39:26.493646+0000","last_became_peered":"2026-02-20T11:39:26.493646+0000","last_unstale":"2026-02-20T11:41:51.140994+0000","last_undegraded":"2026-02-20T11:41:51.140994+0000","last_fullsized":"2026-02-20T11:41:51.140994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":157,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126578+0000","last_change":"2026-02-20T11:39:28.448056+0000","last_active":"2026-02-20T11:41:51.126578+0000","last_peered":"2026-02-20T11:41:51.126578+0000","last_clean":"2026-02-20T11:41:51.126578+0000","last_became_active":"2026-02-20T11:39:26.463577+0000","last_became_peered":"2026-02-20T11:39:26.463577+0000","last_unstale":"2026-02-20T11:41:51.126578+0000","last_undegraded":"2026-02-20T11:41:51.126578+0000","last_fullsized":"2026-02-20T11:41:51.126578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":231,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.125529+0000","last_change":"2026-02-20T11:39:22.453510+0000","last_active":"2026-02-20T11:41:51.125529+0000","last_peered":"2026-02-20T11:41:51.125529+0000","last_clean":"2026-02-20T11:41:51.125529+0000","last_became_active":"2026-02-20T11:39:22.444155+0000","last_became_peered":"2026-02-20T11:39:22.444155+0000","last_unstale":"2026-02-20T11:41:51.125529+0000","last_undegraded":"2026-02-20T11:41:51.125529+0000","last_fullsized":"2026-02-20T11:41:51.125529+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_clean_scrub_stamp":"2026-02-20T11:39:21.408836+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160245235712,"internally_reserved":0,"allocated":10731520,"data_stored":10731520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":0,"up_from":10,"seq":42949673114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3804,"kb_used_data":3804,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162852,"statfs":{"total":53418655744,"available":53414760448,"internally_reserved":0,"allocated":3895296,"data_stored":3895296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640410,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3836,"kb_used_data":3836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162820,"statfs":{"total":53418655744,"available":53414727680,"internally_reserved":0,"allocated":3928064,"data_stored":3928064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705818,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":2644,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52164012,"statfs":{"total":53418655744,"available":53415948288,"internally_reserved":0,"allocated":2707456,"data_stored":2707456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53414563840,"internally_reserved":0,"allocated":4091904,"data_stored":4091904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53415948288,"internally_reserved":0,"allocated":2707456,"data_stored":2707456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:41:57.239 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:41:57.314 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:57.310+0000 7fadb385a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:57.316 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:57.314+0000 7fadb385a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:57.428 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:41:57.428 INFO:teuthology.orchestra.run.trial127.stderr:dumped all 2026-02-20T11:41:57.438 INFO:teuthology.orchestra.run.trial127.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2026-02-20T11:41:57.385157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":10284,"kb_used_data":10284,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156489684,"statfs":{"total":160255967232,"available":160245436416,"internally_reserved":0,"allocated":10530816,"data_stored":10530816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.274420"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126452+0000","last_change":"2026-02-20T11:39:28.445207+0000","last_active":"2026-02-20T11:41:51.126452+0000","last_peered":"2026-02-20T11:41:51.126452+0000","last_clean":"2026-02-20T11:41:51.126452+0000","last_became_active":"2026-02-20T11:39:26.458667+0000","last_became_peered":"2026-02-20T11:39:26.458667+0000","last_unstale":"2026-02-20T11:41:51.126452+0000","last_undegraded":"2026-02-20T11:41:51.126452+0000","last_fullsized":"2026-02-20T11:41:51.126452+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126320+0000","last_change":"2026-02-20T11:39:28.445864+0000","last_active":"2026-02-20T11:41:51.126320+0000","last_peered":"2026-02-20T11:41:51.126320+0000","last_clean":"2026-02-20T11:41:51.126320+0000","last_became_active":"2026-02-20T11:39:26.468015+0000","last_became_peered":"2026-02-20T11:39:26.468015+0000","last_unstale":"2026-02-20T11:41:51.126320+0000","last_undegraded":"2026-02-20T11:41:51.126320+0000","last_fullsized":"2026-02-20T11:41:51.126320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.128542+0000","last_change":"2026-02-20T11:39:28.448158+0000","last_active":"2026-02-20T11:41:51.128542+0000","last_peered":"2026-02-20T11:41:51.128542+0000","last_clean":"2026-02-20T11:41:51.128542+0000","last_became_active":"2026-02-20T11:39:26.478805+0000","last_became_peered":"2026-02-20T11:39:26.478805+0000","last_unstale":"2026-02-20T11:41:51.128542+0000","last_undegraded":"2026-02-20T11:41:51.128542+0000","last_fullsized":"2026-02-20T11:41:51.128542+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.127689+0000","last_change":"2026-02-20T11:39:28.445859+0000","last_active":"2026-02-20T11:41:51.127689+0000","last_peered":"2026-02-20T11:41:51.127689+0000","last_clean":"2026-02-20T11:41:51.127689+0000","last_became_active":"2026-02-20T11:39:26.472032+0000","last_became_peered":"2026-02-20T11:39:26.472032+0000","last_unstale":"2026-02-20T11:41:51.127689+0000","last_undegraded":"2026-02-20T11:41:51.127689+0000","last_fullsized":"2026-02-20T11:41:51.127689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":158,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126559+0000","last_change":"2026-02-20T11:39:28.444205+0000","last_active":"2026-02-20T11:41:51.126559+0000","last_peered":"2026-02-20T11:41:51.126559+0000","last_clean":"2026-02-20T11:41:51.126559+0000","last_became_active":"2026-02-20T11:39:26.472432+0000","last_became_peered":"2026-02-20T11:39:26.472432+0000","last_unstale":"2026-02-20T11:41:51.126559+0000","last_undegraded":"2026-02-20T11:41:51.126559+0000","last_fullsized":"2026-02-20T11:41:51.126559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.143152+0000","last_change":"2026-02-20T11:39:28.443886+0000","last_active":"2026-02-20T11:41:51.143152+0000","last_peered":"2026-02-20T11:41:51.143152+0000","last_clean":"2026-02-20T11:41:51.143152+0000","last_became_active":"2026-02-20T11:39:26.472666+0000","last_became_peered":"2026-02-20T11:39:26.472666+0000","last_unstale":"2026-02-20T11:41:51.143152+0000","last_undegraded":"2026-02-20T11:41:51.143152+0000","last_fullsized":"2026-02-20T11:41:51.143152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.140994+0000","last_change":"2026-02-20T11:39:28.442460+0000","last_active":"2026-02-20T11:41:51.140994+0000","last_peered":"2026-02-20T11:41:51.140994+0000","last_clean":"2026-02-20T11:41:51.140994+0000","last_became_active":"2026-02-20T11:39:26.493646+0000","last_became_peered":"2026-02-20T11:39:26.493646+0000","last_unstale":"2026-02-20T11:41:51.140994+0000","last_undegraded":"2026-02-20T11:41:51.140994+0000","last_fullsized":"2026-02-20T11:41:51.140994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":157,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126578+0000","last_change":"2026-02-20T11:39:28.448056+0000","last_active":"2026-02-20T11:41:51.126578+0000","last_peered":"2026-02-20T11:41:51.126578+0000","last_clean":"2026-02-20T11:41:51.126578+0000","last_became_active":"2026-02-20T11:39:26.463577+0000","last_became_peered":"2026-02-20T11:39:26.463577+0000","last_unstale":"2026-02-20T11:41:51.126578+0000","last_undegraded":"2026-02-20T11:41:51.126578+0000","last_fullsized":"2026-02-20T11:41:51.126578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":231,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.125529+0000","last_change":"2026-02-20T11:39:22.453510+0000","last_active":"2026-02-20T11:41:51.125529+0000","last_peered":"2026-02-20T11:41:51.125529+0000","last_clean":"2026-02-20T11:41:51.125529+0000","last_became_active":"2026-02-20T11:39:22.444155+0000","last_became_peered":"2026-02-20T11:39:22.444155+0000","last_unstale":"2026-02-20T11:41:51.125529+0000","last_undegraded":"2026-02-20T11:41:51.125529+0000","last_fullsized":"2026-02-20T11:41:51.125529+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_clean_scrub_stamp":"2026-02-20T11:39:21.408836+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160245235712,"internally_reserved":0,"allocated":10731520,"data_stored":10731520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":0,"up_from":10,"seq":42949673114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3804,"kb_used_data":3804,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162852,"statfs":{"total":53418655744,"available":53414760448,"internally_reserved":0,"allocated":3895296,"data_stored":3895296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640410,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3836,"kb_used_data":3836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162820,"statfs":{"total":53418655744,"available":53414727680,"internally_reserved":0,"allocated":3928064,"data_stored":3928064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705818,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":2644,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52164012,"statfs":{"total":53418655744,"available":53415948288,"internally_reserved":0,"allocated":2707456,"data_stored":2707456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53414563840,"internally_reserved":0,"allocated":4091904,"data_stored":4091904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53415948288,"internally_reserved":0,"allocated":2707456,"data_stored":2707456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:41:57.439 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-20T11:41:57.439 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:41:57.515 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:57.511+0000 7f7650b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:57.517 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:57.515+0000 7f7650b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:57.628 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:41:57.629 INFO:teuthology.orchestra.run.trial127.stderr:dumped all 2026-02-20T11:41:57.638 INFO:teuthology.orchestra.run.trial127.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2026-02-20T11:41:57.385157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":10284,"kb_used_data":10284,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156489684,"statfs":{"total":160255967232,"available":160245436416,"internally_reserved":0,"allocated":10530816,"data_stored":10530816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.274420"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126452+0000","last_change":"2026-02-20T11:39:28.445207+0000","last_active":"2026-02-20T11:41:51.126452+0000","last_peered":"2026-02-20T11:41:51.126452+0000","last_clean":"2026-02-20T11:41:51.126452+0000","last_became_active":"2026-02-20T11:39:26.458667+0000","last_became_peered":"2026-02-20T11:39:26.458667+0000","last_unstale":"2026-02-20T11:41:51.126452+0000","last_undegraded":"2026-02-20T11:41:51.126452+0000","last_fullsized":"2026-02-20T11:41:51.126452+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126320+0000","last_change":"2026-02-20T11:39:28.445864+0000","last_active":"2026-02-20T11:41:51.126320+0000","last_peered":"2026-02-20T11:41:51.126320+0000","last_clean":"2026-02-20T11:41:51.126320+0000","last_became_active":"2026-02-20T11:39:26.468015+0000","last_became_peered":"2026-02-20T11:39:26.468015+0000","last_unstale":"2026-02-20T11:41:51.126320+0000","last_undegraded":"2026-02-20T11:41:51.126320+0000","last_fullsized":"2026-02-20T11:41:51.126320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.128542+0000","last_change":"2026-02-20T11:39:28.448158+0000","last_active":"2026-02-20T11:41:51.128542+0000","last_peered":"2026-02-20T11:41:51.128542+0000","last_clean":"2026-02-20T11:41:51.128542+0000","last_became_active":"2026-02-20T11:39:26.478805+0000","last_became_peered":"2026-02-20T11:39:26.478805+0000","last_unstale":"2026-02-20T11:41:51.128542+0000","last_undegraded":"2026-02-20T11:41:51.128542+0000","last_fullsized":"2026-02-20T11:41:51.128542+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.127689+0000","last_change":"2026-02-20T11:39:28.445859+0000","last_active":"2026-02-20T11:41:51.127689+0000","last_peered":"2026-02-20T11:41:51.127689+0000","last_clean":"2026-02-20T11:41:51.127689+0000","last_became_active":"2026-02-20T11:39:26.472032+0000","last_became_peered":"2026-02-20T11:39:26.472032+0000","last_unstale":"2026-02-20T11:41:51.127689+0000","last_undegraded":"2026-02-20T11:41:51.127689+0000","last_fullsized":"2026-02-20T11:41:51.127689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":158,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126559+0000","last_change":"2026-02-20T11:39:28.444205+0000","last_active":"2026-02-20T11:41:51.126559+0000","last_peered":"2026-02-20T11:41:51.126559+0000","last_clean":"2026-02-20T11:41:51.126559+0000","last_became_active":"2026-02-20T11:39:26.472432+0000","last_became_peered":"2026-02-20T11:39:26.472432+0000","last_unstale":"2026-02-20T11:41:51.126559+0000","last_undegraded":"2026-02-20T11:41:51.126559+0000","last_fullsized":"2026-02-20T11:41:51.126559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.143152+0000","last_change":"2026-02-20T11:39:28.443886+0000","last_active":"2026-02-20T11:41:51.143152+0000","last_peered":"2026-02-20T11:41:51.143152+0000","last_clean":"2026-02-20T11:41:51.143152+0000","last_became_active":"2026-02-20T11:39:26.472666+0000","last_became_peered":"2026-02-20T11:39:26.472666+0000","last_unstale":"2026-02-20T11:41:51.143152+0000","last_undegraded":"2026-02-20T11:41:51.143152+0000","last_fullsized":"2026-02-20T11:41:51.143152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.140994+0000","last_change":"2026-02-20T11:39:28.442460+0000","last_active":"2026-02-20T11:41:51.140994+0000","last_peered":"2026-02-20T11:41:51.140994+0000","last_clean":"2026-02-20T11:41:51.140994+0000","last_became_active":"2026-02-20T11:39:26.493646+0000","last_became_peered":"2026-02-20T11:39:26.493646+0000","last_unstale":"2026-02-20T11:41:51.140994+0000","last_undegraded":"2026-02-20T11:41:51.140994+0000","last_fullsized":"2026-02-20T11:41:51.140994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":157,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126578+0000","last_change":"2026-02-20T11:39:28.448056+0000","last_active":"2026-02-20T11:41:51.126578+0000","last_peered":"2026-02-20T11:41:51.126578+0000","last_clean":"2026-02-20T11:41:51.126578+0000","last_became_active":"2026-02-20T11:39:26.463577+0000","last_became_peered":"2026-02-20T11:39:26.463577+0000","last_unstale":"2026-02-20T11:41:51.126578+0000","last_undegraded":"2026-02-20T11:41:51.126578+0000","last_fullsized":"2026-02-20T11:41:51.126578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":231,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.125529+0000","last_change":"2026-02-20T11:39:22.453510+0000","last_active":"2026-02-20T11:41:51.125529+0000","last_peered":"2026-02-20T11:41:51.125529+0000","last_clean":"2026-02-20T11:41:51.125529+0000","last_became_active":"2026-02-20T11:39:22.444155+0000","last_became_peered":"2026-02-20T11:39:22.444155+0000","last_unstale":"2026-02-20T11:41:51.125529+0000","last_undegraded":"2026-02-20T11:41:51.125529+0000","last_fullsized":"2026-02-20T11:41:51.125529+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_clean_scrub_stamp":"2026-02-20T11:39:21.408836+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160245235712,"internally_reserved":0,"allocated":10731520,"data_stored":10731520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":0,"up_from":10,"seq":42949673114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3804,"kb_used_data":3804,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162852,"statfs":{"total":53418655744,"available":53414760448,"internally_reserved":0,"allocated":3895296,"data_stored":3895296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640410,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3836,"kb_used_data":3836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162820,"statfs":{"total":53418655744,"available":53414727680,"internally_reserved":0,"allocated":3928064,"data_stored":3928064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705818,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":2644,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52164012,"statfs":{"total":53418655744,"available":53415948288,"internally_reserved":0,"allocated":2707456,"data_stored":2707456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53414563840,"internally_reserved":0,"allocated":4091904,"data_stored":4091904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53415948288,"internally_reserved":0,"allocated":2707456,"data_stored":2707456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:41:57.639 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:41:57.715 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:57.711+0000 7f5da0d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:57.717 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:57.715+0000 7f5da0d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:57.831 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:41:57.831 INFO:teuthology.orchestra.run.trial127.stdout:{"epoch":66,"fsid":"f7dd0554-3a23-4321-9783-f9664d26aacc","created":"2026-02-20T11:39:08.893977+0000","modified":"2026-02-20T11:41:56.436756+0000","last_up_change":"2026-02-20T11:39:20.403763+0000","last_in_change":"2026-02-20T11:39:10.663202+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:39:21.373294+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T11:39:24.632123+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"982f21a5-e2eb-4f51-b6cf-27356488c525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6803","nonce":928988629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6804","nonce":928988629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6812","nonce":928988629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6811","nonce":928988629}]},"public_addr":"10.20.193.127:6803/928988629","cluster_addr":"10.20.193.127:6804/928988629","heartbeat_back_addr":"10.20.193.127:6812/928988629","heartbeat_front_addr":"10.20.193.127:6811/928988629","state":["exists","up"]},{"osd":1,"uuid":"2ff1b0af-eb05-4858-b8ce-5d3581c2527f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6807","nonce":2643657845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6808","nonce":2643657845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6806","nonce":2643657845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6813","nonce":2643657845}]},"public_addr":"10.20.193.127:6807/2643657845","cluster_addr":"10.20.193.127:6808/2643657845","heartbeat_back_addr":"10.20.193.127:6806/2643657845","heartbeat_front_addr":"10.20.193.127:6813/2643657845","state":["exists","up"]},{"osd":2,"uuid":"1cc9a239-0369-4885-b55c-f3046a325f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6810","nonce":3657503840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6809","nonce":3657503840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6805","nonce":3657503840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.127:6802","nonce":3657503840}]},"public_addr":"10.20.193.127:6810/3657503840","cluster_addr":"10.20.193.127:6809/3657503840","heartbeat_back_addr":"10.20.193.127:6805/3657503840","heartbeat_front_addr":"10.20.193.127:6802/3657503840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]},{"pool":3,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:41:58.842 INFO:tasks.ceph:Scrubbing osd.0 2026-02-20T11:41:58.843 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-20T11:41:58.926 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:58.918+0000 7fdb8e28b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:58.928 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:58.926+0000 7fdb8e28b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:58.936 INFO:teuthology.orchestra.run.trial127.stdout:{ 2026-02-20T11:41:58.937 INFO:teuthology.orchestra.run.trial127.stdout: "success": "" 2026-02-20T11:41:58.937 INFO:teuthology.orchestra.run.trial127.stdout:} 2026-02-20T11:41:58.944 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-20T11:41:59.022 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.017+0000 7efd5b0c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.024 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.022+0000 7efd5b0c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.135 INFO:teuthology.orchestra.run.trial127.stderr:instructed osd(s) 0 to deep-scrub 2026-02-20T11:41:59.146 INFO:tasks.ceph:Scrubbing osd.1 2026-02-20T11:41:59.146 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-20T11:41:59.226 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.220+0000 7f6e8f3dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.228 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.226+0000 7f6e8f3dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.238 INFO:teuthology.orchestra.run.trial127.stdout:{ 2026-02-20T11:41:59.238 INFO:teuthology.orchestra.run.trial127.stdout: "success": "" 2026-02-20T11:41:59.238 INFO:teuthology.orchestra.run.trial127.stdout:} 2026-02-20T11:41:59.246 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-20T11:41:59.324 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.319+0000 7f49bd70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.326 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.324+0000 7f49bd70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.439 INFO:teuthology.orchestra.run.trial127.stderr:instructed osd(s) 1 to deep-scrub 2026-02-20T11:41:59.451 INFO:tasks.ceph:Scrubbing osd.2 2026-02-20T11:41:59.452 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-20T11:41:59.533 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.526+0000 7f5f0ed93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.535 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.533+0000 7f5f0ed93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.545 INFO:teuthology.orchestra.run.trial127.stdout:{ 2026-02-20T11:41:59.545 INFO:teuthology.orchestra.run.trial127.stdout: "success": "" 2026-02-20T11:41:59.545 INFO:teuthology.orchestra.run.trial127.stdout:} 2026-02-20T11:41:59.554 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-20T11:41:59.631 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.626+0000 7f5d0aa5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.633 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.631+0000 7f5d0aa5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.747 INFO:teuthology.orchestra.run.trial127.stderr:instructed osd(s) 2 to deep-scrub 2026-02-20T11:41:59.759 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:41:59.836 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.831+0000 7fc3fbe5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.838 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:41:59.836+0000 7fc3fbe5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:41:59.948 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:41:59.948 INFO:teuthology.orchestra.run.trial127.stderr:dumped all 2026-02-20T11:41:59.958 INFO:teuthology.orchestra.run.trial127.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2026-02-20T11:41:59.385389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":8696,"kb_used_data":8696,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156491272,"statfs":{"total":160255967232,"available":160247062528,"internally_reserved":0,"allocated":8904704,"data_stored":8904704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001099"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126452+0000","last_change":"2026-02-20T11:39:28.445207+0000","last_active":"2026-02-20T11:41:51.126452+0000","last_peered":"2026-02-20T11:41:51.126452+0000","last_clean":"2026-02-20T11:41:51.126452+0000","last_became_active":"2026-02-20T11:39:26.458667+0000","last_became_peered":"2026-02-20T11:39:26.458667+0000","last_unstale":"2026-02-20T11:41:51.126452+0000","last_undegraded":"2026-02-20T11:41:51.126452+0000","last_fullsized":"2026-02-20T11:41:51.126452+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126320+0000","last_change":"2026-02-20T11:39:28.445864+0000","last_active":"2026-02-20T11:41:51.126320+0000","last_peered":"2026-02-20T11:41:51.126320+0000","last_clean":"2026-02-20T11:41:51.126320+0000","last_became_active":"2026-02-20T11:39:26.468015+0000","last_became_peered":"2026-02-20T11:39:26.468015+0000","last_unstale":"2026-02-20T11:41:51.126320+0000","last_undegraded":"2026-02-20T11:41:51.126320+0000","last_fullsized":"2026-02-20T11:41:51.126320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.128542+0000","last_change":"2026-02-20T11:39:28.448158+0000","last_active":"2026-02-20T11:41:51.128542+0000","last_peered":"2026-02-20T11:41:51.128542+0000","last_clean":"2026-02-20T11:41:51.128542+0000","last_became_active":"2026-02-20T11:39:26.478805+0000","last_became_peered":"2026-02-20T11:39:26.478805+0000","last_unstale":"2026-02-20T11:41:51.128542+0000","last_undegraded":"2026-02-20T11:41:51.128542+0000","last_fullsized":"2026-02-20T11:41:51.128542+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":156,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.127689+0000","last_change":"2026-02-20T11:39:28.445859+0000","last_active":"2026-02-20T11:41:51.127689+0000","last_peered":"2026-02-20T11:41:51.127689+0000","last_clean":"2026-02-20T11:41:51.127689+0000","last_became_active":"2026-02-20T11:39:26.472032+0000","last_became_peered":"2026-02-20T11:39:26.472032+0000","last_unstale":"2026-02-20T11:41:51.127689+0000","last_undegraded":"2026-02-20T11:41:51.127689+0000","last_fullsized":"2026-02-20T11:41:51.127689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":158,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126559+0000","last_change":"2026-02-20T11:39:28.444205+0000","last_active":"2026-02-20T11:41:51.126559+0000","last_peered":"2026-02-20T11:41:51.126559+0000","last_clean":"2026-02-20T11:41:51.126559+0000","last_became_active":"2026-02-20T11:39:26.472432+0000","last_became_peered":"2026-02-20T11:39:26.472432+0000","last_unstale":"2026-02-20T11:41:51.126559+0000","last_undegraded":"2026-02-20T11:41:51.126559+0000","last_fullsized":"2026-02-20T11:41:51.126559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":159,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-20T11:41:56.469146+0000","last_change":"2026-02-20T11:39:28.443886+0000","last_active":"2026-02-20T11:41:56.469146+0000","last_peered":"2026-02-20T11:41:56.469146+0000","last_clean":"2026-02-20T11:41:56.469146+0000","last_became_active":"2026-02-20T11:39:26.472666+0000","last_became_peered":"2026-02-20T11:39:26.472666+0000","last_unstale":"2026-02-20T11:41:56.469146+0000","last_undegraded":"2026-02-20T11:41:56.469146+0000","last_fullsized":"2026-02-20T11:41:56.469146+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-20T11:41:56.460237+0000","last_change":"2026-02-20T11:39:28.442460+0000","last_active":"2026-02-20T11:41:56.460237+0000","last_peered":"2026-02-20T11:41:56.460237+0000","last_clean":"2026-02-20T11:41:56.460237+0000","last_became_active":"2026-02-20T11:39:26.493646+0000","last_became_peered":"2026-02-20T11:39:26.493646+0000","last_unstale":"2026-02-20T11:41:56.460237+0000","last_undegraded":"2026-02-20T11:41:56.460237+0000","last_fullsized":"2026-02-20T11:41:56.460237+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":157,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.126578+0000","last_change":"2026-02-20T11:39:28.448056+0000","last_active":"2026-02-20T11:41:51.126578+0000","last_peered":"2026-02-20T11:41:51.126578+0000","last_clean":"2026-02-20T11:41:51.126578+0000","last_became_active":"2026-02-20T11:39:26.463577+0000","last_became_peered":"2026-02-20T11:39:26.463577+0000","last_unstale":"2026-02-20T11:41:51.126578+0000","last_undegraded":"2026-02-20T11:41:51.126578+0000","last_fullsized":"2026-02-20T11:41:51.126578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:25.418832+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":231,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-20T11:41:51.125529+0000","last_change":"2026-02-20T11:39:22.453510+0000","last_active":"2026-02-20T11:41:51.125529+0000","last_peered":"2026-02-20T11:41:51.125529+0000","last_clean":"2026-02-20T11:41:51.125529+0000","last_became_active":"2026-02-20T11:39:22.444155+0000","last_became_peered":"2026-02-20T11:39:22.444155+0000","last_unstale":"2026-02-20T11:41:51.125529+0000","last_undegraded":"2026-02-20T11:41:51.125529+0000","last_fullsized":"2026-02-20T11:41:51.125529+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:39:21.408836+0000","last_clean_scrub_stamp":"2026-02-20T11:39:21.408836+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160246861824,"internally_reserved":0,"allocated":9105408,"data_stored":9105408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":0,"up_from":10,"seq":42949673114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3804,"kb_used_data":3804,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162852,"statfs":{"total":53418655744,"available":53414760448,"internally_reserved":0,"allocated":3895296,"data_stored":3895296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640410,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3836,"kb_used_data":3836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162820,"statfs":{"total":53418655744,"available":53414727680,"internally_reserved":0,"allocated":3928064,"data_stored":3928064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705823,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1056,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165600,"statfs":{"total":53418655744,"available":53417574400,"internally_reserved":0,"allocated":1081344,"data_stored":1081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53414563840,"internally_reserved":0,"allocated":4091904,"data_stored":4091904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53414723584,"internally_reserved":0,"allocated":3932160,"data_stored":3932160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53417574400,"internally_reserved":0,"allocated":1081344,"data_stored":1081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:41:59.961 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-20T11:39:25.418832+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=39, tm_sec=25, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=41, tm_sec=57, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:41:59.961 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-20T11:39:25.418832+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=39, tm_sec=25, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=41, tm_sec=57, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:41:59.961 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-20T11:39:25.418832+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=39, tm_sec=25, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=41, tm_sec=57, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:41:59.961 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-20T11:39:25.418832+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=39, tm_sec=25, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=41, tm_sec=57, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:41:59.961 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-20T11:39:25.418832+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=39, tm_sec=25, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=41, tm_sec=57, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:41:59.961 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-20T11:39:25.418832+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=39, tm_sec=25, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=41, tm_sec=57, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:41:59.961 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-20T11:39:25.418832+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=39, tm_sec=25, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=41, tm_sec=57, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:41:59.961 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-20T11:39:25.418832+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=39, tm_sec=25, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=41, tm_sec=57, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:41:59.961 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-20T11:39:21.408836+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=39, tm_sec=21, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=11, tm_min=41, tm_sec=57, tm_wday=4, tm_yday=51, tm_isdst=0) 2026-02-20T11:41:59.961 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-20T11:42:19.962 DEBUG:teuthology.orchestra.run.trial127:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:42:20.041 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:42:20.036+0000 7f7d08a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:20.043 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:42:20.041+0000 7f7d08a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:20.153 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-20T11:42:20.153 INFO:teuthology.orchestra.run.trial127.stderr:dumped all 2026-02-20T11:42:20.164 INFO:teuthology.orchestra.run.trial127.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2026-02-20T11:42:19.388856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":6012,"kb_used_data":6012,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156493956,"statfs":{"total":160255967232,"available":160249810944,"internally_reserved":0,"allocated":6156288,"data_stored":6156288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001506"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":173,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-20T11:42:13.925832+0000","last_change":"2026-02-20T11:41:59.455596+0000","last_active":"2026-02-20T11:42:13.925832+0000","last_peered":"2026-02-20T11:42:13.925832+0000","last_clean":"2026-02-20T11:42:13.925832+0000","last_became_active":"2026-02-20T11:39:26.458667+0000","last_became_peered":"2026-02-20T11:39:26.458667+0000","last_unstale":"2026-02-20T11:42:13.925832+0000","last_undegraded":"2026-02-20T11:42:13.925832+0000","last_fullsized":"2026-02-20T11:42:13.925832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:41:59.455545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:41:59.455545+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":173,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-20T11:42:13.927047+0000","last_change":"2026-02-20T11:41:59.451379+0000","last_active":"2026-02-20T11:42:13.927047+0000","last_peered":"2026-02-20T11:42:13.927047+0000","last_clean":"2026-02-20T11:42:13.927047+0000","last_became_active":"2026-02-20T11:39:26.468015+0000","last_became_peered":"2026-02-20T11:39:26.468015+0000","last_unstale":"2026-02-20T11:42:13.927047+0000","last_undegraded":"2026-02-20T11:42:13.927047+0000","last_fullsized":"2026-02-20T11:42:13.927047+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:41:59.451323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:41:59.451323+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-20T11:42:13.926834+0000","last_change":"2026-02-20T11:41:59.454268+0000","last_active":"2026-02-20T11:42:13.926834+0000","last_peered":"2026-02-20T11:42:13.926834+0000","last_clean":"2026-02-20T11:42:13.926834+0000","last_became_active":"2026-02-20T11:39:26.478805+0000","last_became_peered":"2026-02-20T11:39:26.478805+0000","last_unstale":"2026-02-20T11:42:13.926834+0000","last_undegraded":"2026-02-20T11:42:13.926834+0000","last_fullsized":"2026-02-20T11:42:13.926834+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:41:59.454217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:41:59.454217+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":173,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-20T11:42:13.926843+0000","last_change":"2026-02-20T11:41:59.450113+0000","last_active":"2026-02-20T11:42:13.926843+0000","last_peered":"2026-02-20T11:42:13.926843+0000","last_clean":"2026-02-20T11:42:13.926843+0000","last_became_active":"2026-02-20T11:39:26.472032+0000","last_became_peered":"2026-02-20T11:39:26.472032+0000","last_unstale":"2026-02-20T11:42:13.926843+0000","last_undegraded":"2026-02-20T11:42:13.926843+0000","last_fullsized":"2026-02-20T11:42:13.926843+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:41:59.450032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:41:59.450032+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":175,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-20T11:42:13.928024+0000","last_change":"2026-02-20T11:41:59.145363+0000","last_active":"2026-02-20T11:42:13.928024+0000","last_peered":"2026-02-20T11:42:13.928024+0000","last_clean":"2026-02-20T11:42:13.928024+0000","last_became_active":"2026-02-20T11:39:26.472432+0000","last_became_peered":"2026-02-20T11:39:26.472432+0000","last_unstale":"2026-02-20T11:42:13.928024+0000","last_undegraded":"2026-02-20T11:42:13.928024+0000","last_fullsized":"2026-02-20T11:42:13.928024+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'2","last_scrub_stamp":"2026-02-20T11:41:59.145286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:41:59.145286+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-20T11:42:13.925850+0000","last_change":"2026-02-20T11:41:59.755067+0000","last_active":"2026-02-20T11:42:13.925850+0000","last_peered":"2026-02-20T11:42:13.925850+0000","last_clean":"2026-02-20T11:42:13.925850+0000","last_became_active":"2026-02-20T11:39:26.472666+0000","last_became_peered":"2026-02-20T11:39:26.472666+0000","last_unstale":"2026-02-20T11:42:13.925850+0000","last_undegraded":"2026-02-20T11:42:13.925850+0000","last_fullsized":"2026-02-20T11:42:13.925850+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:41:59.754982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:41:59.754982+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":173,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-20T11:42:13.924996+0000","last_change":"2026-02-20T11:41:59.754419+0000","last_active":"2026-02-20T11:42:13.924996+0000","last_peered":"2026-02-20T11:42:13.924996+0000","last_clean":"2026-02-20T11:42:13.924996+0000","last_became_active":"2026-02-20T11:39:26.493646+0000","last_became_peered":"2026-02-20T11:39:26.493646+0000","last_unstale":"2026-02-20T11:42:13.924996+0000","last_undegraded":"2026-02-20T11:42:13.924996+0000","last_fullsized":"2026-02-20T11:42:13.924996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:41:59.754358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:41:59.754358+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":174,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-20T11:42:13.927228+0000","last_change":"2026-02-20T11:41:59.452572+0000","last_active":"2026-02-20T11:42:13.927228+0000","last_peered":"2026-02-20T11:42:13.927228+0000","last_clean":"2026-02-20T11:42:13.927228+0000","last_became_active":"2026-02-20T11:39:26.463577+0000","last_became_peered":"2026-02-20T11:39:26.463577+0000","last_unstale":"2026-02-20T11:42:13.927228+0000","last_undegraded":"2026-02-20T11:42:13.927228+0000","last_fullsized":"2026-02-20T11:42:13.927228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2026-02-20T11:41:59.452506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:41:59.452506+0000","last_clean_scrub_stamp":"2026-02-20T11:39:25.418832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":248,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-20T11:42:13.925545+0000","last_change":"2026-02-20T11:41:59.466349+0000","last_active":"2026-02-20T11:42:13.925545+0000","last_peered":"2026-02-20T11:42:13.925545+0000","last_clean":"2026-02-20T11:42:13.925545+0000","last_became_active":"2026-02-20T11:39:22.444155+0000","last_became_peered":"2026-02-20T11:39:22.444155+0000","last_unstale":"2026-02-20T11:42:13.925545+0000","last_undegraded":"2026-02-20T11:42:13.925545+0000","last_fullsized":"2026-02-20T11:42:13.925545+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"13'52","last_scrub_stamp":"2026-02-20T11:41:59.466279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:41:59.466279+0000","last_clean_scrub_stamp":"2026-02-20T11:39:21.408836+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160249802752,"internally_reserved":0,"allocated":6164480,"data_stored":6164480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53416124416,"internally_reserved":0,"allocated":2531328,"data_stored":2531328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":0,"up_from":10,"seq":42949673134,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":2452,"kb_used_data":2452,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52164204,"statfs":{"total":53418655744,"available":53416144896,"internally_reserved":0,"allocated":2510848,"data_stored":2510848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640430,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":2472,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52164184,"statfs":{"total":53418655744,"available":53416124416,"internally_reserved":0,"allocated":2531328,"data_stored":2531328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1088,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165568,"statfs":{"total":53418655744,"available":53417541632,"internally_reserved":0,"allocated":1114112,"data_stored":1114112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53416124416,"internally_reserved":0,"allocated":2531328,"data_stored":2531328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53416136704,"internally_reserved":0,"allocated":2519040,"data_stored":2519040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53416124416,"internally_reserved":0,"allocated":2531328,"data_stored":2531328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53417541632,"internally_reserved":0,"allocated":1114112,"data_stored":1114112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:42:20.165 DEBUG:teuthology.orchestra.run.trial127:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-20T11:42:20.242 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:42:20.235+0000 7f50d1f9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:20.244 INFO:teuthology.orchestra.run.trial127.stderr:2026-02-20T11:42:20.242+0000 7f50d1f9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:42:20.371 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-20T11:42:20.371 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-20T11:42:20.372 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-20T11:42:20.372 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T11:42:20.443 INFO:tasks.ceph.osd.0:Stopped 2026-02-20T11:42:20.444 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-20T11:42:20.444 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T11:42:20.519 INFO:tasks.ceph.osd.1:Stopped 2026-02-20T11:42:20.519 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-20T11:42:20.519 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T11:42:20.587 INFO:tasks.ceph.osd.2:Stopped 2026-02-20T11:42:20.587 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-20T11:42:20.588 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-20T11:42:20.588 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T11:42:20.622 INFO:tasks.ceph.mgr.x:Stopped 2026-02-20T11:42:20.623 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-20T11:42:20.623 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-20T11:42:20.623 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T11:42:20.673 INFO:tasks.ceph.mon.a:Stopped 2026-02-20T11:42:20.673 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-20T11:42:20.673 DEBUG:teuthology.orchestra.run.trial127:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2026-02-20T11:42:20.739 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial127.front.sepia.ceph.com 2026-02-20T11:42:20.740 DEBUG:teuthology.orchestra.run.trial127:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-20T11:42:20.807 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial127.front.sepia.ceph.com 2026-02-20T11:42:20.808 DEBUG:teuthology.orchestra.run.trial127:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-20T11:42:20.875 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial127.front.sepia.ceph.com 2026-02-20T11:42:20.875 DEBUG:teuthology.orchestra.run.trial127:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-20T11:42:20.948 INFO:tasks.ceph:Archiving mon data... 2026-02-20T11:42:20.950 DEBUG:teuthology.misc:Transferring archived files from trial127:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60490/data/mon.a.tgz 2026-02-20T11:42:20.950 DEBUG:teuthology.orchestra.run.trial127:> mktemp 2026-02-20T11:42:21.005 INFO:teuthology.orchestra.run.trial127.stdout:/tmp/tmp.hRd595ZCDY 2026-02-20T11:42:21.005 DEBUG:teuthology.orchestra.run.trial127:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.hRd595ZCDY 2026-02-20T11:42:21.096 DEBUG:teuthology.orchestra.run.trial127:> sudo chmod 0666 /tmp/tmp.hRd595ZCDY 2026-02-20T11:42:21.177 DEBUG:teuthology.orchestra.remote:trial127:/tmp/tmp.hRd595ZCDY is 232KB 2026-02-20T11:42:21.266 DEBUG:teuthology.orchestra.run.trial127:> rm -fr /tmp/tmp.hRd595ZCDY 2026-02-20T11:42:21.283 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-20T11:42:21.284 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-20T11:42:21.404 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-20T11:42:21.404 INFO:tasks.ceph:Archiving crash dumps... 2026-02-20T11:42:21.406 DEBUG:teuthology.misc:Transferring archived files from trial127:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60490/remote/trial127/crash 2026-02-20T11:42:21.407 DEBUG:teuthology.orchestra.run.trial127:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-20T11:42:21.468 INFO:tasks.ceph:Compressing logs... 2026-02-20T11:42:21.468 DEBUG:teuthology.orchestra.run.trial127:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-20T11:42:21.532 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-20T11:42:21.532 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42208.log 2026-02-20T11:42:21.533 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-20T11:42:21.533 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.42208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41674.log 2026-02-20T11:42:21.533 INFO:teuthology.orchestra.run.trial127.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.42208.log.gz 2026-02-20T11:42:21.533 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41522.log 2026-02-20T11:42:21.533 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43482.log 2026-02-20T11:42:21.533 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41674.log.gz 2026-02-20T11:42:21.534 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41522.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41522.log.gz 2026-02-20T11:42:21.534 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-20T11:42:21.534 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.43482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40412.log 2026-02-20T11:42:21.534 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43482.log.gz 2026-02-20T11:42:21.534 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41914.log 2026-02-20T11:42:21.534 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.40412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43578.log 2026-02-20T11:42:21.535 INFO:teuthology.orchestra.run.trial127.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.40412.log.gz 2026-02-20T11:42:21.535 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41914.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41914.log.gz 2026-02-20T11:42:21.535 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43531.log 2026-02-20T11:42:21.535 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.43578.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43578.log.gz 2026-02-20T11:42:21.535 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41473.log 2026-02-20T11:42:21.535 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.43531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43433.log 2026-02-20T11:42:21.535 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43531.log.gz 2026-02-20T11:42:21.536 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41473.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41473.log.gz 2026-02-20T11:42:21.536 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42849.log 2026-02-20T11:42:21.536 INFO:teuthology.orchestra.run.trial127.stderr:gzip/var/log/ceph/ceph-client.admin.43433.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41673.log 2026-02-20T11:42:21.536 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43433.log.gz 2026-02-20T11:42:21.536 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.42849.log: 50.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42849.log.gz -5 2026-02-20T11:42:21.536 INFO:teuthology.orchestra.run.trial127.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40785.log 2026-02-20T11:42:21.536 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42555.log 2026-02-20T11:42:21.536 INFO:teuthology.orchestra.run.trial127.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41673.log.gz 2026-02-20T11:42:21.537 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.40785.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40785.log.gzgzip 2026-02-20T11:42:21.537 INFO:teuthology.orchestra.run.trial127.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41034.log 2026-02-20T11:42:21.537 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.42555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41571.log 2026-02-20T11:42:21.537 INFO:teuthology.orchestra.run.trial127.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42555.log.gz 2026-02-20T11:42:21.537 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41263.log 2026-02-20T11:42:21.537 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41034.log.gz 2026-02-20T11:42:21.538 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43090.log 2026-02-20T11:42:21.538 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41571.log.gz 2026-02-20T11:42:21.538 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41263.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41263.log.gz 2026-02-20T11:42:21.538 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41333.log 2026-02-20T11:42:21.538 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.43090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41671.log 2026-02-20T11:42:21.538 INFO:teuthology.orchestra.run.trial127.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.43090.log.gz 2026-02-20T11:42:21.538 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41333.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41333.log.gz 2026-02-20T11:42:21.538 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41424.log 2026-02-20T11:42:21.539 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41671.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 50.0% 2026-02-20T11:42:21.539 INFO:teuthology.orchestra.run.trial127.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41671.log.gz 2026-02-20T11:42:21.539 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41424.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41424.log.gz 2026-02-20T11:42:21.539 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42159.log 2026-02-20T11:42:21.539 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43139.log 2026-02-20T11:42:21.539 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.42159.log: 87.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 49.6% 2026-02-20T11:42:21.539 INFO:teuthology.orchestra.run.trial127.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42159.log.gz 2026-02-20T11:42:21.539 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41941.log 2026-02-20T11:42:21.540 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.43139.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.43139.log.gz 2026-02-20T11:42:21.540 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42992.log 2026-02-20T11:42:21.540 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41941.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41941.log.gzgzip 2026-02-20T11:42:21.540 INFO:teuthology.orchestra.run.trial127.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-20T11:42:21.540 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.42992.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42992.log.gz 2026-02-20T11:42:21.540 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40567.log 2026-02-20T11:42:21.540 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43384.log 2026-02-20T11:42:21.540 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.40567.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40567.log.gz 2026-02-20T11:42:21.541 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41819.log 2026-02-20T11:42:21.541 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.43384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43041.log 2026-02-20T11:42:21.541 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43384.log.gz 2026-02-20T11:42:21.541 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41819.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41819.log.gz 2026-02-20T11:42:21.541 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41359.log 2026-02-20T11:42:21.541 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.43041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42110.log 2026-02-20T11:42:21.541 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43041.log.gz 2026-02-20T11:42:21.542 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41359.log: 66.1% -- replaced with /var/log/ceph/ceph-client.admin.41359.log.gz 2026-02-20T11:42:21.542 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43237.log 2026-02-20T11:42:21.542 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.42110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42012.log 2026-02-20T11:42:21.542 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42110.log.gz 2026-02-20T11:42:21.542 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.43237.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43237.log.gz 2026-02-20T11:42:21.542 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42482.log 2026-02-20T11:42:21.542 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.42012.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.39124.log 2026-02-20T11:42:21.542 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42012.log.gz 2026-02-20T11:42:21.543 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.42482.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.42482.log.gz 2026-02-20T11:42:21.543 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43286.log 2026-02-20T11:42:21.543 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph.tmp-client.admin.39124.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-20T11:42:21.543 INFO:teuthology.orchestra.run.trial127.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.39124.log.gz 2026-02-20T11:42:21.543 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.43286.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43286.log.gz 2026-02-20T11:42:21.543 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42039.log 2026-02-20T11:42:21.543 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41821.log 2026-02-20T11:42:21.544 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.42039.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.42039.log.gz 2026-02-20T11:42:21.544 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43335.log 2026-02-20T11:42:21.544 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40507.log 2026-02-20T11:42:21.544 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41821.log.gz 2026-02-20T11:42:21.544 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.43335.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43335.log.gz 2026-02-20T11:42:21.544 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40738.log 2026-02-20T11:42:21.544 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.40507.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-20T11:42:21.545 INFO:teuthology.orchestra.run.trial127.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40507.log.gz 2026-02-20T11:42:21.545 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.40738.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40738.log.gz 2026-02-20T11:42:21.545 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40691.log 2026-02-20T11:42:21.545 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41004.log 2026-02-20T11:42:21.545 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.40691.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40691.log.gz 2026-02-20T11:42:21.545 INFO:teuthology.orchestra.run.trial127.stderr: 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-20T11:42:21.545 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43188.log 2026-02-20T11:42:21.546 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41820.log 2026-02-20T11:42:21.546 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41004.log.gz 2026-02-20T11:42:21.546 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.43188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41193.log 2026-02-20T11:42:21.546 INFO:teuthology.orchestra.run.trial127.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43188.log.gz 2026-02-20T11:42:21.546 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41820.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41820.log.gz 2026-02-20T11:42:21.546 INFO:teuthology.orchestra.run.trial127.stderr:/var/log/ceph/ceph-client.admin.41193.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41193.log.gz 2026-02-20T11:42:21.556 INFO:teuthology.orchestra.run.trial127.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-20T11:42:21.592 INFO:teuthology.orchestra.run.trial127.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-20T11:42:22.672 INFO:teuthology.orchestra.run.trial127.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-20T11:42:23.051 INFO:teuthology.orchestra.run.trial127.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-20T11:42:23.613 INFO:teuthology.orchestra.run.trial127.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-20T11:42:23.615 INFO:teuthology.orchestra.run.trial127.stderr: 2026-02-20T11:42:23.616 INFO:teuthology.orchestra.run.trial127.stderr:real 0m2.091s 2026-02-20T11:42:23.616 INFO:teuthology.orchestra.run.trial127.stderr:user 0m4.564s 2026-02-20T11:42:23.616 INFO:teuthology.orchestra.run.trial127.stderr:sys 0m0.276s 2026-02-20T11:42:23.616 INFO:tasks.ceph:Archiving logs... 2026-02-20T11:42:23.616 DEBUG:teuthology.misc:Transferring archived files from trial127:/var/log/ceph to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60490/remote/trial127/log 2026-02-20T11:42:23.618 DEBUG:teuthology.orchestra.run.trial127:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-20T11:42:24.244 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-20T11:42:24.570 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-20T11:42:24.581 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-20T11:42:24.581 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-20T11:42:24.607 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-20T11:42:24.622 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-20T11:42:24.623 DEBUG:teuthology.orchestra.run.trial127:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T11:42:24.663 INFO:teuthology.orchestra.run.trial127.stderr:bash: line 1: ntpq: command not found 2026-02-20T11:42:25.026 INFO:teuthology.orchestra.run.trial127.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T11:42:25.026 INFO:teuthology.orchestra.run.trial127.stdout:=============================================================================== 2026-02-20T11:42:25.026 INFO:teuthology.orchestra.run.trial127.stdout:^- 24-111-79-186-dynamic.mi> 2 6 377 60 +431us[ +576us] +/- 80ms 2026-02-20T11:42:25.026 INFO:teuthology.orchestra.run.trial127.stdout:^- frumentum-zeus.infra.fru> 2 6 377 61 -2778us[-2634us] +/- 25ms 2026-02-20T11:42:25.026 INFO:teuthology.orchestra.run.trial127.stdout:^* 50.205.57.38 1 6 377 58 -65us[ +80us] +/- 5167us 2026-02-20T11:42:25.026 INFO:teuthology.orchestra.run.trial127.stdout:^- 38.45.64.130 2 6 377 59 +175us[ +320us] +/- 17ms 2026-02-20T11:42:25.028 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-20T11:42:25.047 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-20T11:42:25.047 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-20T11:42:25.063 DEBUG:teuthology.orchestra.run.trial127:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-20T11:42:25.122 DEBUG:teuthology.orchestra.run.trial127:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T11:42:25.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:42:25.150 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-20T11:42:25.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-20T11:42:25.182 INFO:teuthology.task.internal:Duration was 878.475976 seconds 2026-02-20T11:42:25.182 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-20T11:42:25.197 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-20T11:42:25.198 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-20T11:42:25.234 INFO:teuthology.orchestra.run.trial127.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T11:42:25.430 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-20T11:42:25.430 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial127.front.sepia.ceph.com 2026-02-20T11:42:25.431 DEBUG:teuthology.orchestra.run.trial127:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-20T11:42:25.490 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-20T11:42:25.490 DEBUG:teuthology.orchestra.run.trial127:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-20T11:42:25.592 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-20T11:42:25.592 DEBUG:teuthology.orchestra.run.trial127:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-20T11:42:25.614 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T11:42:25.614 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T11:42:25.615 INFO:teuthology.orchestra.run.trial127.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-20T11:42:25.615 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-20T11:42:25.615 INFO:teuthology.orchestra.run.trial127.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-20T11:42:25.625 INFO:teuthology.orchestra.run.trial127.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-20T11:42:25.627 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-20T11:42:25.644 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-20T11:42:25.644 DEBUG:teuthology.orchestra.run.trial127:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-20T11:42:25.691 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-20T11:42:25.716 DEBUG:teuthology.orchestra.run.trial127:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-20T11:42:25.755 INFO:teuthology.orchestra.run.trial127.stdout:kernel.core_pattern = core 2026-02-20T11:42:25.773 DEBUG:teuthology.orchestra.run.trial127:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-20T11:42:25.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:42:25.830 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-20T11:42:25.863 INFO:teuthology.task.internal:Transferring archived files... 2026-02-20T11:42:25.864 DEBUG:teuthology.misc:Transferring archived files from trial127:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60490/remote/trial127 2026-02-20T11:42:25.864 DEBUG:teuthology.orchestra.run.trial127:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-20T11:42:25.906 INFO:teuthology.task.internal:Removing archive directory... 2026-02-20T11:42:25.906 DEBUG:teuthology.orchestra.run.trial127:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-20T11:42:25.954 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-20T11:42:25.972 INFO:teuthology.task.internal:Not uploading archives. 2026-02-20T11:42:25.973 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-20T11:42:25.989 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-20T11:42:25.990 DEBUG:teuthology.orchestra.run.trial127:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-20T11:42:26.012 INFO:teuthology.orchestra.run.trial127.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 20 11:42 /home/ubuntu/cephtest 2026-02-20T11:42:26.013 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-20T11:42:26.024 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-20T11:42:26.065 INFO:teuthology.run:Summary data: description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rbd_cls_tests} duration: 878.4759764671326 flavor: debug owner: scheduled_jjperez@soko04.front.sepia.ceph.com success: true 2026-02-20T11:42:26.065 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T11:42:26.264 INFO:teuthology.run:pass