2026-02-26T03:04:33.518 INFO:root:teuthology version: 1.2.4.dev11+g6e3d0419c 2026-02-26T03:04:33.533 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-26T03:04:33.611 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71166 branch: main description: crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rados_python} email: null first_in_suite: false flavor: debug job_id: '71166' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(PG_ - \(OSD_ - \(OBJECT_ - \(POOL_APP_NOT_ENABLED\) sha1: abffb617683951511fccf3c3cab6e219f2ec83a8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_system_packages: - python3-pytest flavor: debug sha1: abffb617683951511fccf3c3cab6e219f2ec83a8 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996 owner: scheduled_shraddhaag@soko04.front.sepia.ceph.com priority: 101 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 5117 sha1: abffb617683951511fccf3c3cab6e219f2ec83a8 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996 targets: trial012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIBq26EaQGQhBUimtfMSyEGHSYLQfmIxPjLSFKjYJdpPFfx1cKeWkRqLO68Vmz2AqfUrmsKMTuInDcDcl4xcH1k= trial068.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN1b3gHvHZqKqXUNBfS4WCaAsVMlt++lKdFheIAW97jHl8QvLS3hEbXawzsN1S9hBsvyVqW8v5EMhNjyr0rjSw4= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 - workunit: clients: client.0: - rados/test_python.sh -m 'not (wait or tier or ec)' timeout: 1h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6e3d0419ccca0e75a03e08feb708946e198236f4 timestamp: 2026-02-26_02:22:56 tube: trial user: shraddhaag verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-26T03:04:33.611 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996/qa; will attempt to use it 2026-02-26T03:04:33.611 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996/qa/tasks 2026-02-26T03:04:33.611 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-26T03:04:33.613 INFO:teuthology.task.internal:Checking packages... 2026-02-26T03:04:33.613 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash 'abffb617683951511fccf3c3cab6e219f2ec83a8' 2026-02-26T03:04:33.613 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-26T03:04:33.613 INFO:teuthology.packaging:ref: None 2026-02-26T03:04:33.613 INFO:teuthology.packaging:tag: None 2026-02-26T03:04:33.613 INFO:teuthology.packaging:branch: main 2026-02-26T03:04:33.613 INFO:teuthology.packaging:sha1: abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:04:33.613 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=main 2026-02-26T03:04:33.744 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5593.g63656871 2026-02-26T03:04:33.746 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-26T03:04:33.758 INFO:teuthology.task.internal:no buildpackages task found 2026-02-26T03:04:33.758 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-26T03:04:33.772 INFO:teuthology.task.internal:Saving configuration 2026-02-26T03:04:33.780 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-26T03:04:33.792 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-26T03:04:33.874 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial012.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71166', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-26 03:01:06.368448', 'locked_by': 'scheduled_shraddhaag@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIBq26EaQGQhBUimtfMSyEGHSYLQfmIxPjLSFKjYJdpPFfx1cKeWkRqLO68Vmz2AqfUrmsKMTuInDcDcl4xcH1k='} 2026-02-26T03:04:33.932 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial068.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71166', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-26 03:01:06.367667', 'locked_by': 'scheduled_shraddhaag@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN1b3gHvHZqKqXUNBfS4WCaAsVMlt++lKdFheIAW97jHl8QvLS3hEbXawzsN1S9hBsvyVqW8v5EMhNjyr0rjSw4='} 2026-02-26T03:04:33.932 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-26T03:04:33.944 INFO:teuthology.task.internal:roles: ubuntu@trial012.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2026-02-26T03:04:33.944 INFO:teuthology.task.internal:roles: ubuntu@trial068.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2026-02-26T03:04:33.944 INFO:teuthology.run_tasks:Running task console_log... 2026-02-26T03:04:34.077 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0dfc157600>, signals=[15]) 2026-02-26T03:04:34.078 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-26T03:04:34.089 INFO:teuthology.task.internal:Opening connections... 2026-02-26T03:04:34.089 DEBUG:teuthology.task.internal:connecting to ubuntu@trial012.front.sepia.ceph.com 2026-02-26T03:04:34.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:04:34.151 DEBUG:teuthology.task.internal:connecting to ubuntu@trial068.front.sepia.ceph.com 2026-02-26T03:04:34.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial068.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:04:34.214 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-26T03:04:34.226 DEBUG:teuthology.orchestra.run.trial012:> uname -m 2026-02-26T03:04:34.242 INFO:teuthology.orchestra.run.trial012.stdout:x86_64 2026-02-26T03:04:34.243 DEBUG:teuthology.orchestra.run.trial012:> cat /etc/os-release 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:NAME="CentOS Stream" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:VERSION="9" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:ID="centos" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:ID_LIKE="rhel fedora" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:VERSION_ID="9" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:PLATFORM_ID="platform:el9" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:ANSI_COLOR="0;31" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:LOGO="fedora-logo-icon" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:HOME_URL="https://centos.org/" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-26T03:04:34.298 INFO:teuthology.orchestra.run.trial012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-26T03:04:34.299 INFO:teuthology.lock.ops:Updating trial012.front.sepia.ceph.com on lock server 2026-02-26T03:04:34.374 DEBUG:teuthology.orchestra.run.trial068:> uname -m 2026-02-26T03:04:34.389 INFO:teuthology.orchestra.run.trial068.stdout:x86_64 2026-02-26T03:04:34.389 DEBUG:teuthology.orchestra.run.trial068:> cat /etc/os-release 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:NAME="CentOS Stream" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:VERSION="9" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:ID="centos" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:ID_LIKE="rhel fedora" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:VERSION_ID="9" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:PLATFORM_ID="platform:el9" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:ANSI_COLOR="0;31" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:LOGO="fedora-logo-icon" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:HOME_URL="https://centos.org/" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-26T03:04:34.445 INFO:teuthology.orchestra.run.trial068.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-26T03:04:34.445 INFO:teuthology.lock.ops:Updating trial068.front.sepia.ceph.com on lock server 2026-02-26T03:04:34.515 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-26T03:04:34.528 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-26T03:04:34.540 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-26T03:04:34.540 DEBUG:teuthology.orchestra.run.trial012:> test '!' -e /home/ubuntu/cephtest 2026-02-26T03:04:34.542 DEBUG:teuthology.orchestra.run.trial068:> test '!' -e /home/ubuntu/cephtest 2026-02-26T03:04:34.557 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-26T03:04:34.568 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-26T03:04:34.568 DEBUG:teuthology.orchestra.run.trial012:> test -z $(ls -A /var/lib/ceph) 2026-02-26T03:04:34.597 DEBUG:teuthology.orchestra.run.trial068:> test -z $(ls -A /var/lib/ceph) 2026-02-26T03:04:34.614 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-26T03:04:34.754 INFO:teuthology.run_tasks:Running task kernel... 2026-02-26T03:04:34.774 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-26T03:04:34.774 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-26T03:04:34.774 DEBUG:teuthology.orchestra.run.trial012:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-26T03:04:34.775 DEBUG:teuthology.orchestra.run.trial068:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-26T03:04:34.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:04:34.790 DEBUG:teuthology.orchestra.run.trial012:> uname -r 2026-02-26T03:04:34.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:04:34.791 DEBUG:teuthology.orchestra.run.trial068:> uname -r 2026-02-26T03:04:34.845 INFO:teuthology.orchestra.run.trial012.stdout:5.14.0-681.el9.x86_64 2026-02-26T03:04:34.846 INFO:teuthology.task.kernel:Running kernel on trial012: 5.14.0-681.el9.x86_64 2026-02-26T03:04:34.846 DEBUG:teuthology.orchestra.run.trial012:> sudo yum install -y kernel 2026-02-26T03:04:34.846 INFO:teuthology.orchestra.run.trial068.stdout:5.14.0-681.el9.x86_64 2026-02-26T03:04:34.846 INFO:teuthology.task.kernel:Running kernel on trial068: 5.14.0-681.el9.x86_64 2026-02-26T03:04:34.846 DEBUG:teuthology.orchestra.run.trial068:> sudo yum install -y kernel 2026-02-26T03:04:36.997 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.9 MB 00:01 2026-02-26T03:04:40.037 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-26T03:04:51.182 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - CRB 932 kB/s | 8.0 MB 00:08 2026-02-26T03:05:56.257 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - BaseOS 112 kB/s | 8.9 MB 01:21 2026-02-26T03:05:59.635 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - AppStream 9.5 MB/s | 27 MB 00:02 2026-02-26T03:06:02.843 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - Extras packages 291 B/s | 20 kB 01:10 2026-02-26T03:06:03.593 INFO:teuthology.orchestra.run.trial068.stdout:Extra Packages for Enterprise Linux 30 MB/s | 20 MB 00:00 2026-02-26T03:06:06.843 INFO:teuthology.orchestra.run.trial068.stdout:lab-extras 1.6 MB/s | 50 kB 00:00 2026-02-26T03:06:07.868 INFO:teuthology.orchestra.run.trial068.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-26T03:06:07.868 INFO:teuthology.orchestra.run.trial068.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-26T03:06:07.889 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-26T03:06:07.889 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-26T03:06:07.889 INFO:teuthology.orchestra.run.trial068.stdout: Package Architecture Version Repository Size 2026-02-26T03:06:07.889 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-26T03:06:07.889 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-26T03:06:07.889 INFO:teuthology.orchestra.run.trial068.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-26T03:06:07.889 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-26T03:06:07.889 INFO:teuthology.orchestra.run.trial068.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-26T03:06:07.889 INFO:teuthology.orchestra.run.trial068.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-26T03:06:07.890 INFO:teuthology.orchestra.run.trial068.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-26T03:06:07.890 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:06:07.890 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-26T03:06:07.890 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-26T03:06:07.890 INFO:teuthology.orchestra.run.trial068.stdout:Install 4 Packages 2026-02-26T03:06:07.890 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:06:07.890 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 89 M 2026-02-26T03:06:07.890 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 135 M 2026-02-26T03:06:07.890 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-26T03:06:08.339 INFO:teuthology.orchestra.run.trial068.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 4.4 MB/s | 901 kB 00:00 2026-02-26T03:06:08.498 INFO:teuthology.orchestra.run.trial068.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 48 MB/s | 17 MB 00:00 2026-02-26T03:06:08.669 INFO:teuthology.orchestra.run.trial068.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 94 MB/s | 31 MB 00:00 2026-02-26T03:06:08.759 INFO:teuthology.orchestra.run.trial068.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 64 MB/s | 40 MB 00:00 2026-02-26T03:06:08.760 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-26T03:06:08.760 INFO:teuthology.orchestra.run.trial068.stdout:Total 102 MB/s | 89 MB 00:00 2026-02-26T03:06:08.950 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-26T03:06:09.151 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-26T03:06:09.151 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-26T03:06:09.536 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-26T03:06:09.537 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-26T03:06:10.654 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-26T03:06:10.946 INFO:teuthology.orchestra.run.trial068.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-26T03:06:10.968 INFO:teuthology.orchestra.run.trial068.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:06:12.079 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - CRB 809 kB/s | 8.0 MB 00:10 2026-02-26T03:06:13.529 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:06:13.563 INFO:teuthology.orchestra.run.trial068.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:06:13.656 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 20 kB 00:00 2026-02-26T03:06:14.612 INFO:teuthology.orchestra.run.trial012.stdout:Extra Packages for Enterprise Linux 23 MB/s | 20 MB 00:00 2026-02-26T03:06:16.604 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:06:16.610 INFO:teuthology.orchestra.run.trial068.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:06:17.800 INFO:teuthology.orchestra.run.trial012.stdout:lab-extras 1.6 MB/s | 50 kB 00:00 2026-02-26T03:06:18.827 INFO:teuthology.orchestra.run.trial012.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-26T03:06:18.827 INFO:teuthology.orchestra.run.trial012.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-26T03:06:18.847 INFO:teuthology.orchestra.run.trial012.stdout:Dependencies resolved. 2026-02-26T03:06:18.848 INFO:teuthology.orchestra.run.trial012.stdout:================================================================================ 2026-02-26T03:06:18.848 INFO:teuthology.orchestra.run.trial012.stdout: Package Architecture Version Repository Size 2026-02-26T03:06:18.848 INFO:teuthology.orchestra.run.trial012.stdout:================================================================================ 2026-02-26T03:06:18.848 INFO:teuthology.orchestra.run.trial012.stdout:Installing: 2026-02-26T03:06:18.848 INFO:teuthology.orchestra.run.trial012.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-26T03:06:18.848 INFO:teuthology.orchestra.run.trial012.stdout:Installing dependencies: 2026-02-26T03:06:18.848 INFO:teuthology.orchestra.run.trial012.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-26T03:06:18.848 INFO:teuthology.orchestra.run.trial012.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-26T03:06:18.849 INFO:teuthology.orchestra.run.trial012.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-26T03:06:18.849 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:06:18.849 INFO:teuthology.orchestra.run.trial012.stdout:Transaction Summary 2026-02-26T03:06:18.849 INFO:teuthology.orchestra.run.trial012.stdout:================================================================================ 2026-02-26T03:06:18.849 INFO:teuthology.orchestra.run.trial012.stdout:Install 4 Packages 2026-02-26T03:06:18.849 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:06:18.849 INFO:teuthology.orchestra.run.trial012.stdout:Total download size: 89 M 2026-02-26T03:06:18.849 INFO:teuthology.orchestra.run.trial012.stdout:Installed size: 135 M 2026-02-26T03:06:18.849 INFO:teuthology.orchestra.run.trial012.stdout:Downloading Packages: 2026-02-26T03:06:19.657 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:06:37.160 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:06:37.181 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:06:37.365 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:06:37.366 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-26T03:06:37.366 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:06:37.366 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:06:37.423 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:06:37.423 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:06:37.423 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-26T03:06:37.424 INFO:teuthology.orchestra.run.trial068.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-26T03:06:37.424 INFO:teuthology.orchestra.run.trial068.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-26T03:06:37.424 INFO:teuthology.orchestra.run.trial068.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-26T03:06:37.424 INFO:teuthology.orchestra.run.trial068.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-26T03:06:37.424 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:06:37.424 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-26T03:06:37.487 DEBUG:teuthology.orchestra.run.trial068:> rpm -q kernel | sort -rV | head -n 1 2026-02-26T03:06:37.516 INFO:teuthology.orchestra.run.trial068.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-26T03:06:37.517 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-26T03:06:37.517 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-26T03:06:37.518 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-26T03:06:37.518 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-26T03:06:37.518 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2026-02-26T03:06:37.518 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial068.front.sepia.ceph.com, path=None, version=distro) 2026-02-26T03:06:37.518 DEBUG:teuthology.orchestra.run.trial068:> rpm -q kernel | sort -rV | head -n 1 2026-02-26T03:06:37.583 INFO:teuthology.orchestra.run.trial068.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-26T03:06:37.583 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-26T03:06:37.584 DEBUG:teuthology.orchestra.run.trial068:> sudo rpm -qi grub2-tools 2026-02-26T03:06:37.651 INFO:teuthology.orchestra.run.trial068.stdout:Name : grub2-tools 2026-02-26T03:06:37.651 INFO:teuthology.orchestra.run.trial068.stdout:Epoch : 1 2026-02-26T03:06:37.651 INFO:teuthology.orchestra.run.trial068.stdout:Version : 2.06 2026-02-26T03:06:37.651 INFO:teuthology.orchestra.run.trial068.stdout:Release : 120.el9 2026-02-26T03:06:37.651 INFO:teuthology.orchestra.run.trial068.stdout:Architecture: x86_64 2026-02-26T03:06:37.651 INFO:teuthology.orchestra.run.trial068.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-26T03:06:37.651 INFO:teuthology.orchestra.run.trial068.stdout:Group : Unspecified 2026-02-26T03:06:37.652 INFO:teuthology.orchestra.run.trial068.stdout:Size : 9108611 2026-02-26T03:06:37.652 INFO:teuthology.orchestra.run.trial068.stdout:License : GPLv3+ 2026-02-26T03:06:37.652 INFO:teuthology.orchestra.run.trial068.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-26T03:06:37.652 INFO:teuthology.orchestra.run.trial068.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-26T03:06:37.652 INFO:teuthology.orchestra.run.trial068.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-26T03:06:37.652 INFO:teuthology.orchestra.run.trial068.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-26T03:06:37.652 INFO:teuthology.orchestra.run.trial068.stdout:Packager : builder@centos.org 2026-02-26T03:06:37.652 INFO:teuthology.orchestra.run.trial068.stdout:Vendor : CentOS 2026-02-26T03:06:37.652 INFO:teuthology.orchestra.run.trial068.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-26T03:06:37.652 INFO:teuthology.orchestra.run.trial068.stdout:Summary : Support tools for GRUB. 2026-02-26T03:06:37.652 INFO:teuthology.orchestra.run.trial068.stdout:Description : 2026-02-26T03:06:37.653 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:06:37.653 INFO:teuthology.orchestra.run.trial068.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-26T03:06:37.653 INFO:teuthology.orchestra.run.trial068.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-26T03:06:37.653 INFO:teuthology.orchestra.run.trial068.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-26T03:06:37.653 INFO:teuthology.orchestra.run.trial068.stdout:hardware devices. 2026-02-26T03:06:37.653 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:06:37.653 INFO:teuthology.orchestra.run.trial068.stdout:This subpackage provides tools for support of all platforms. 2026-02-26T03:06:37.654 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-26T03:06:37.654 INFO:teuthology.task.kernel:Updating grub on trial068 to boot 5.14.0-686.el9.x86_64 2026-02-26T03:06:37.655 DEBUG:teuthology.orchestra.run.trial068:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-26T03:06:37.719 INFO:teuthology.orchestra.run.trial068.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-26T03:06:37.721 DEBUG:teuthology.orchestra.run.trial068:> sudo command -v grubby && echo yes || echo no 2026-02-26T03:06:37.787 INFO:teuthology.orchestra.run.trial068.stdout:/sbin/grubby 2026-02-26T03:06:37.788 INFO:teuthology.orchestra.run.trial068.stdout:yes 2026-02-26T03:06:37.789 DEBUG:teuthology.orchestra.run.trial068:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-26T03:06:37.853 INFO:teuthology.orchestra.run.trial068.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-26T03:06:37.854 DEBUG:teuthology.orchestra.run.trial068:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-26T03:06:37.929 DEBUG:teuthology.orchestra.run.trial068:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-26T03:06:38.133 INFO:teuthology.orchestra.run.trial068.stderr:Generating grub configuration file ... 2026-02-26T03:06:38.200 INFO:teuthology.orchestra.run.trial068.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-26T03:06:38.206 INFO:teuthology.orchestra.run.trial068.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-26T03:06:38.453 INFO:teuthology.orchestra.run.trial068.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-26T03:06:38.460 INFO:teuthology.orchestra.run.trial068.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-26T03:06:38.494 INFO:teuthology.orchestra.run.trial068.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-26T03:06:38.500 INFO:teuthology.orchestra.run.trial068.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-26T03:06:38.531 INFO:teuthology.orchestra.run.trial068.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-26T03:06:38.537 INFO:teuthology.orchestra.run.trial068.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-26T03:06:38.561 INFO:teuthology.orchestra.run.trial068.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-26T03:06:38.568 INFO:teuthology.orchestra.run.trial068.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-26T03:06:38.653 INFO:teuthology.orchestra.run.trial068.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-26T03:06:38.661 INFO:teuthology.orchestra.run.trial068.stderr:done 2026-02-26T03:06:38.663 DEBUG:teuthology.orchestra.run.trial068:> test -d /sys/firmware/efi 2026-02-26T03:06:38.679 DEBUG:teuthology.orchestra.run.trial068:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-26T03:06:38.744 INFO:teuthology.orchestra.run.trial068.stdout:/boot/efi/EFI/BOOT 2026-02-26T03:06:38.744 INFO:teuthology.orchestra.run.trial068.stdout:/boot/efi/EFI/centos 2026-02-26T03:06:38.745 DEBUG:teuthology.orchestra.run.trial068:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-26T03:06:38.810 INFO:teuthology.orchestra.run.trial068.stdout:no 2026-02-26T03:06:38.811 DEBUG:teuthology.orchestra.run.trial068:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-26T03:06:38.874 INFO:teuthology.orchestra.run.trial068.stdout:yes 2026-02-26T03:06:38.875 DEBUG:teuthology.orchestra.run.trial068:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-26T03:06:38.940 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial068 2026-02-26T03:06:38.941 DEBUG:teuthology.orchestra.run.trial068:> sudo bash -c ' 2026-02-26T03:06:38.941 DEBUG:teuthology.orchestra.run.trial068:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-26T03:06:38.941 DEBUG:teuthology.orchestra.run.trial068:> ' 2026-02-26T03:06:48.230 INFO:teuthology.orchestra.run.trial012.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 31 kB/s | 901 kB 00:29 2026-02-26T03:06:48.510 INFO:teuthology.orchestra.run.trial012.stdout:(2/4): kernel-modules-core-5.14.0-686.el9.x86_6 111 MB/s | 31 MB 00:00 2026-02-26T03:06:49.411 INFO:teuthology.orchestra.run.trial012.stdout:(3/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 1.3 MB/s | 40 MB 00:30 2026-02-26T03:06:49.499 INFO:teuthology.orchestra.run.trial012.stdout:(4/4): kernel-core-5.14.0-686.el9.x86_64.rpm 575 kB/s | 17 MB 00:30 2026-02-26T03:06:49.499 INFO:teuthology.orchestra.run.trial012.stdout:-------------------------------------------------------------------------------- 2026-02-26T03:06:49.499 INFO:teuthology.orchestra.run.trial012.stdout:Total 2.9 MB/s | 89 MB 00:30 2026-02-26T03:06:49.691 INFO:teuthology.orchestra.run.trial012.stdout:Running transaction check 2026-02-26T03:06:49.892 INFO:teuthology.orchestra.run.trial012.stdout:Transaction check succeeded. 2026-02-26T03:06:49.892 INFO:teuthology.orchestra.run.trial012.stdout:Running transaction test 2026-02-26T03:06:50.281 INFO:teuthology.orchestra.run.trial012.stdout:Transaction test succeeded. 2026-02-26T03:06:50.282 INFO:teuthology.orchestra.run.trial012.stdout:Running transaction 2026-02-26T03:06:51.399 INFO:teuthology.orchestra.run.trial012.stdout: Preparing : 1/1 2026-02-26T03:06:51.691 INFO:teuthology.orchestra.run.trial012.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-26T03:06:51.713 INFO:teuthology.orchestra.run.trial012.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:06:54.283 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:06:54.317 INFO:teuthology.orchestra.run.trial012.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:06:57.351 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:06:57.356 INFO:teuthology.orchestra.run.trial012.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:07:00.406 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:07:08.983 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2026-02-26T03:07:08.983 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-26T03:07:08.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial068.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:07:17.789 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:07:17.810 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:07:17.996 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:07:17.996 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-26T03:07:17.996 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:07:17.996 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:07:18.053 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:07:18.053 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:07:18.054 INFO:teuthology.orchestra.run.trial012.stdout:Installed: 2026-02-26T03:07:18.054 INFO:teuthology.orchestra.run.trial012.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-26T03:07:18.054 INFO:teuthology.orchestra.run.trial012.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-26T03:07:18.054 INFO:teuthology.orchestra.run.trial012.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-26T03:07:18.054 INFO:teuthology.orchestra.run.trial012.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-26T03:07:18.054 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:07:18.054 INFO:teuthology.orchestra.run.trial012.stdout:Complete! 2026-02-26T03:07:18.116 DEBUG:teuthology.orchestra.run.trial012:> rpm -q kernel | sort -rV | head -n 1 2026-02-26T03:07:18.145 INFO:teuthology.orchestra.run.trial012.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-26T03:07:18.146 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-26T03:07:18.146 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-26T03:07:18.147 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-26T03:07:18.147 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-26T03:07:18.147 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-26T03:07:18.147 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial012.front.sepia.ceph.com, path=None, version=distro) 2026-02-26T03:07:18.147 DEBUG:teuthology.orchestra.run.trial012:> rpm -q kernel | sort -rV | head -n 1 2026-02-26T03:07:18.211 INFO:teuthology.orchestra.run.trial012.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-26T03:07:18.213 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-26T03:07:18.213 DEBUG:teuthology.orchestra.run.trial012:> sudo rpm -qi grub2-tools 2026-02-26T03:07:18.283 INFO:teuthology.orchestra.run.trial012.stdout:Name : grub2-tools 2026-02-26T03:07:18.283 INFO:teuthology.orchestra.run.trial012.stdout:Epoch : 1 2026-02-26T03:07:18.283 INFO:teuthology.orchestra.run.trial012.stdout:Version : 2.06 2026-02-26T03:07:18.283 INFO:teuthology.orchestra.run.trial012.stdout:Release : 120.el9 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:Architecture: x86_64 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:Group : Unspecified 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:Size : 9108611 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:License : GPLv3+ 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:Packager : builder@centos.org 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:Vendor : CentOS 2026-02-26T03:07:18.284 INFO:teuthology.orchestra.run.trial012.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-26T03:07:18.285 INFO:teuthology.orchestra.run.trial012.stdout:Summary : Support tools for GRUB. 2026-02-26T03:07:18.285 INFO:teuthology.orchestra.run.trial012.stdout:Description : 2026-02-26T03:07:18.285 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:07:18.285 INFO:teuthology.orchestra.run.trial012.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-26T03:07:18.285 INFO:teuthology.orchestra.run.trial012.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-26T03:07:18.285 INFO:teuthology.orchestra.run.trial012.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-26T03:07:18.285 INFO:teuthology.orchestra.run.trial012.stdout:hardware devices. 2026-02-26T03:07:18.285 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:07:18.285 INFO:teuthology.orchestra.run.trial012.stdout:This subpackage provides tools for support of all platforms. 2026-02-26T03:07:18.286 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-26T03:07:18.287 INFO:teuthology.task.kernel:Updating grub on trial012 to boot 5.14.0-686.el9.x86_64 2026-02-26T03:07:18.287 DEBUG:teuthology.orchestra.run.trial012:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-26T03:07:18.350 INFO:teuthology.orchestra.run.trial012.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-26T03:07:18.351 DEBUG:teuthology.orchestra.run.trial012:> sudo command -v grubby && echo yes || echo no 2026-02-26T03:07:18.414 INFO:teuthology.orchestra.run.trial012.stdout:/sbin/grubby 2026-02-26T03:07:18.415 INFO:teuthology.orchestra.run.trial012.stdout:yes 2026-02-26T03:07:18.416 DEBUG:teuthology.orchestra.run.trial012:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-26T03:07:18.480 INFO:teuthology.orchestra.run.trial012.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-26T03:07:18.481 DEBUG:teuthology.orchestra.run.trial012:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-26T03:07:18.553 DEBUG:teuthology.orchestra.run.trial012:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-26T03:07:18.758 INFO:teuthology.orchestra.run.trial012.stderr:Generating grub configuration file ... 2026-02-26T03:07:18.827 INFO:teuthology.orchestra.run.trial012.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-26T03:07:18.834 INFO:teuthology.orchestra.run.trial012.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-26T03:07:19.081 INFO:teuthology.orchestra.run.trial012.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-26T03:07:19.088 INFO:teuthology.orchestra.run.trial012.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-26T03:07:19.122 INFO:teuthology.orchestra.run.trial012.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-26T03:07:19.129 INFO:teuthology.orchestra.run.trial012.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-26T03:07:19.160 INFO:teuthology.orchestra.run.trial012.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-26T03:07:19.166 INFO:teuthology.orchestra.run.trial012.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-26T03:07:19.190 INFO:teuthology.orchestra.run.trial012.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-26T03:07:19.196 INFO:teuthology.orchestra.run.trial012.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-26T03:07:19.283 INFO:teuthology.orchestra.run.trial012.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-26T03:07:19.291 INFO:teuthology.orchestra.run.trial012.stderr:done 2026-02-26T03:07:19.293 DEBUG:teuthology.orchestra.run.trial012:> test -d /sys/firmware/efi 2026-02-26T03:07:19.308 DEBUG:teuthology.orchestra.run.trial012:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-26T03:07:19.374 INFO:teuthology.orchestra.run.trial012.stdout:/boot/efi/EFI/BOOT 2026-02-26T03:07:19.374 INFO:teuthology.orchestra.run.trial012.stdout:/boot/efi/EFI/centos 2026-02-26T03:07:19.376 DEBUG:teuthology.orchestra.run.trial012:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-26T03:07:19.440 INFO:teuthology.orchestra.run.trial012.stdout:no 2026-02-26T03:07:19.441 DEBUG:teuthology.orchestra.run.trial012:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-26T03:07:19.505 INFO:teuthology.orchestra.run.trial012.stdout:yes 2026-02-26T03:07:19.505 DEBUG:teuthology.orchestra.run.trial012:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-26T03:07:19.571 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial012 2026-02-26T03:07:19.571 DEBUG:teuthology.orchestra.run.trial012:> sudo bash -c ' 2026-02-26T03:07:19.572 DEBUG:teuthology.orchestra.run.trial012:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-26T03:07:19.572 DEBUG:teuthology.orchestra.run.trial012:> ' 2026-02-26T03:07:49.614 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-26T03:07:49.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial012.front.sepia.ceph.com' 2026-02-26T03:07:49.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:08:09.006 DEBUG:teuthology.orchestra.remote:timed out 2026-02-26T03:08:18.014 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-26T03:08:18.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial068.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:08:18.109 DEBUG:teuthology.orchestra.run.trial068:> true 2026-02-26T03:08:18.265 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-26T03:08:18.265 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-686.el9.x86_64"... 2026-02-26T03:08:18.265 DEBUG:teuthology.orchestra.run.trial068:> uname -r 2026-02-26T03:08:18.322 INFO:teuthology.orchestra.run.trial068.stdout:5.14.0-686.el9.x86_64 2026-02-26T03:08:18.322 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-26T03:08:18.322 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-26T03:08:18.323 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-26T03:08:19.323 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2026-02-26T03:08:19.324 DEBUG:teuthology.orchestra.run.trial068:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-26T03:08:19.354 INFO:teuthology.orchestra.run.trial068.stdout:ttyS1 2026-02-26T03:08:19.389 DEBUG:teuthology.parallel:result is None 2026-02-26T03:08:49.616 DEBUG:teuthology.orchestra.remote:timed out 2026-02-26T03:08:58.618 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial012.front.sepia.ceph.com' 2026-02-26T03:08:58.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:08:58.744 DEBUG:teuthology.orchestra.run.trial012:> true 2026-02-26T03:08:58.896 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial012.front.sepia.ceph.com' 2026-02-26T03:08:58.896 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-686.el9.x86_64"... 2026-02-26T03:08:58.897 DEBUG:teuthology.orchestra.run.trial012:> uname -r 2026-02-26T03:08:58.953 INFO:teuthology.orchestra.run.trial012.stdout:5.14.0-686.el9.x86_64 2026-02-26T03:08:58.953 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-26T03:08:58.954 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-26T03:08:58.954 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-26T03:08:59.954 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-26T03:08:59.955 DEBUG:teuthology.orchestra.run.trial012:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-26T03:08:59.986 INFO:teuthology.orchestra.run.trial012.stdout:ttyS1 2026-02-26T03:09:00.008 DEBUG:teuthology.parallel:result is None 2026-02-26T03:09:00.009 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-26T03:09:00.025 INFO:teuthology.task.internal:Creating test directory... 2026-02-26T03:09:00.025 DEBUG:teuthology.orchestra.run.trial012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-26T03:09:00.051 DEBUG:teuthology.orchestra.run.trial068:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-26T03:09:00.070 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-26T03:09:00.086 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-26T03:09:00.099 INFO:teuthology.task.internal:Creating archive directory... 2026-02-26T03:09:00.100 DEBUG:teuthology.orchestra.run.trial012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-26T03:09:00.108 DEBUG:teuthology.orchestra.run.trial068:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-26T03:09:00.134 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-26T03:09:00.150 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-26T03:09:00.150 DEBUG:teuthology.orchestra.run.trial012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-26T03:09:00.165 DEBUG:teuthology.orchestra.run.trial068:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-26T03:09:00.186 INFO:teuthology.orchestra.run.trial012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:09:00.191 INFO:teuthology.orchestra.run.trial068.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:09:00.195 INFO:teuthology.orchestra.run.trial012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:09:00.199 INFO:teuthology.orchestra.run.trial068.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:09:00.201 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-26T03:09:00.214 INFO:teuthology.task.internal:Configuring sudo... 2026-02-26T03:09:00.214 DEBUG:teuthology.orchestra.run.trial012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-26T03:09:00.241 DEBUG:teuthology.orchestra.run.trial068:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-26T03:09:00.264 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-26T03:09:00.281 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-26T03:09:00.281 DEBUG:teuthology.orchestra.run.trial012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-26T03:09:00.305 DEBUG:teuthology.orchestra.run.trial068:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-26T03:09:00.320 DEBUG:teuthology.orchestra.run.trial012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:09:00.484 DEBUG:teuthology.orchestra.run.trial012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:09:00.509 DEBUG:teuthology.orchestra.run.trial012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:09:00.628 DEBUG:teuthology.orchestra.run.trial012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:09:00.652 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:09:00.652 DEBUG:teuthology.orchestra.run.trial012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-26T03:09:00.715 DEBUG:teuthology.orchestra.run.trial068:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:09:00.820 DEBUG:teuthology.orchestra.run.trial068:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:09:00.845 DEBUG:teuthology.orchestra.run.trial068:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:09:00.967 DEBUG:teuthology.orchestra.run.trial068:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:09:00.989 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:09:00.989 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-26T03:09:01.053 DEBUG:teuthology.orchestra.run.trial012:> sudo service rsyslog restart 2026-02-26T03:09:01.055 DEBUG:teuthology.orchestra.run.trial068:> sudo service rsyslog restart 2026-02-26T03:09:01.081 INFO:teuthology.orchestra.run.trial012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-26T03:09:01.121 INFO:teuthology.orchestra.run.trial068.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-26T03:09:01.610 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-26T03:09:01.623 INFO:teuthology.task.internal:Starting timer... 2026-02-26T03:09:01.623 INFO:teuthology.run_tasks:Running task pcp... 2026-02-26T03:09:01.635 INFO:teuthology.run_tasks:Running task selinux... 2026-02-26T03:09:01.647 DEBUG:teuthology.orchestra.run.trial012:> sudo service auditd rotate 2026-02-26T03:09:01.677 INFO:teuthology.orchestra.run.trial012.stdout:Rotating logs: 2026-02-26T03:09:01.678 DEBUG:teuthology.orchestra.run.trial068:> sudo service auditd rotate 2026-02-26T03:09:01.708 INFO:teuthology.orchestra.run.trial068.stdout:Rotating logs: 2026-02-26T03:09:01.710 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-26T03:09:01.710 DEBUG:teuthology.orchestra.run.trial012:> /usr/sbin/getenforce 2026-02-26T03:09:01.734 INFO:teuthology.orchestra.run.trial012.stdout:Permissive 2026-02-26T03:09:01.734 DEBUG:teuthology.orchestra.run.trial068:> /usr/sbin/getenforce 2026-02-26T03:09:01.766 INFO:teuthology.orchestra.run.trial068.stdout:Permissive 2026-02-26T03:09:01.766 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial012.front.sepia.ceph.com': 'permissive', 'ubuntu@trial068.front.sepia.ceph.com': 'permissive'} 2026-02-26T03:09:01.766 DEBUG:teuthology.orchestra.run.trial012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-26T03:09:01.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:09:01.798 DEBUG:teuthology.orchestra.run.trial068:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-26T03:09:01.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:09:01.831 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-26T03:09:01.831 DEBUG:teuthology.orchestra.run.trial012:> sudo /usr/sbin/setenforce permissive 2026-02-26T03:09:01.861 DEBUG:teuthology.orchestra.run.trial068:> sudo /usr/sbin/setenforce permissive 2026-02-26T03:09:01.895 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-26T03:09:01.907 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-26T03:09:01.915 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-26T03:09:01.915 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial012.front.sepia.ceph.com,trial068.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-26T03:11:47.769 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial012.front.sepia.ceph.com'), Remote(name='ubuntu@trial068.front.sepia.ceph.com')] 2026-02-26T03:11:47.770 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial012.front.sepia.ceph.com' 2026-02-26T03:11:47.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:11:47.923 DEBUG:teuthology.orchestra.run.trial012:> true 2026-02-26T03:11:48.002 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial012.front.sepia.ceph.com' 2026-02-26T03:11:48.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-26T03:11:48.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial068.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:11:48.069 DEBUG:teuthology.orchestra.run.trial068:> true 2026-02-26T03:11:48.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-26T03:11:48.146 INFO:teuthology.run_tasks:Running task clock... 2026-02-26T03:11:48.154 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-26T03:11:48.154 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-26T03:11:48.154 DEBUG:teuthology.orchestra.run.trial012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-26T03:11:48.156 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-26T03:11:48.156 DEBUG:teuthology.orchestra.run.trial068:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-26T03:11:48.182 INFO:teuthology.orchestra.run.trial012.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-26T03:11:48.194 INFO:teuthology.orchestra.run.trial012.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-26T03:11:48.212 INFO:teuthology.orchestra.run.trial068.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-26T03:11:48.215 INFO:teuthology.orchestra.run.trial012.stderr:sudo: ntpd: command not found 2026-02-26T03:11:48.224 INFO:teuthology.orchestra.run.trial068.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-26T03:11:48.224 INFO:teuthology.orchestra.run.trial012.stdout:506 Cannot talk to daemon 2026-02-26T03:11:48.236 INFO:teuthology.orchestra.run.trial012.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-26T03:11:48.244 INFO:teuthology.orchestra.run.trial068.stderr:sudo: ntpd: command not found 2026-02-26T03:11:48.248 INFO:teuthology.orchestra.run.trial012.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-26T03:11:48.254 INFO:teuthology.orchestra.run.trial068.stdout:506 Cannot talk to daemon 2026-02-26T03:11:48.266 INFO:teuthology.orchestra.run.trial068.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-26T03:11:48.278 INFO:teuthology.orchestra.run.trial068.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-26T03:11:48.297 INFO:teuthology.orchestra.run.trial012.stderr:bash: line 1: ntpq: command not found 2026-02-26T03:11:48.319 INFO:teuthology.orchestra.run.trial012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-26T03:11:48.319 INFO:teuthology.orchestra.run.trial012.stdout:=============================================================================== 2026-02-26T03:11:48.319 INFO:teuthology.orchestra.run.trial012.stdout:^? 23-95-35-34-host.colocro> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:11:48.319 INFO:teuthology.orchestra.run.trial012.stdout:^? 23.186.168.131 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:11:48.319 INFO:teuthology.orchestra.run.trial012.stdout:^? B1-66ER.matrix.gs 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:11:48.319 INFO:teuthology.orchestra.run.trial012.stdout:^? 66.42.71.197.vultruserco> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:11:48.322 INFO:teuthology.orchestra.run.trial068.stderr:bash: line 1: ntpq: command not found 2026-02-26T03:11:48.345 INFO:teuthology.orchestra.run.trial068.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-26T03:11:48.345 INFO:teuthology.orchestra.run.trial068.stdout:=============================================================================== 2026-02-26T03:11:48.345 INFO:teuthology.orchestra.run.trial068.stdout:^? 172-234-37-140.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:11:48.345 INFO:teuthology.orchestra.run.trial068.stdout:^? time.trtnw.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:11:48.345 INFO:teuthology.orchestra.run.trial068.stdout:^? ip216-250-115-174.pbiaas> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:11:48.345 INFO:teuthology.orchestra.run.trial068.stdout:^? y.ns.gin.ntt.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:11:48.347 INFO:teuthology.run_tasks:Running task install... 2026-02-26T03:11:48.360 DEBUG:teuthology.task.install:project ceph 2026-02-26T03:11:48.360 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'}} 2026-02-26T03:11:48.360 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'} 2026-02-26T03:11:48.360 INFO:teuthology.task.install:Using flavor: debug 2026-02-26T03:11:48.370 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-26T03:11:48.370 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-26T03:11:48.371 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'tag': None, 'wait_for_package': False} 2026-02-26T03:11:48.372 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:11:48.375 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'tag': None, 'wait_for_package': False} 2026-02-26T03:11:48.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:11:48.574 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/ 2026-02-26T03:11:48.574 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5312.gabffb617 2026-02-26T03:11:48.583 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/ 2026-02-26T03:11:48.583 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5312.gabffb617 2026-02-26T03:11:48.930 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-26T03:11:48.930 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:11:48.930 DEBUG:teuthology.orchestra.run.trial012:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-26T03:11:48.956 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo, python3-pytest on remote rpm x86_64 2026-02-26T03:11:48.956 DEBUG:teuthology.orchestra.run.trial012:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-26T03:11:48.965 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-26T03:11:48.965 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:11:48.965 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-26T03:11:48.992 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo, python3-pytest on remote rpm x86_64 2026-02-26T03:11:48.992 DEBUG:teuthology.orchestra.run.trial068:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-26T03:11:49.021 DEBUG:teuthology.orchestra.run.trial012:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-26T03:11:49.058 DEBUG:teuthology.orchestra.run.trial068:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-26T03:11:49.094 DEBUG:teuthology.orchestra.run.trial012:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-26T03:11:49.133 DEBUG:teuthology.orchestra.run.trial068:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-26T03:11:49.158 INFO:teuthology.orchestra.run.trial012.stdout:check_obsoletes = 1 2026-02-26T03:11:49.159 DEBUG:teuthology.orchestra.run.trial012:> sudo yum clean all 2026-02-26T03:11:49.197 INFO:teuthology.orchestra.run.trial068.stdout:check_obsoletes = 1 2026-02-26T03:11:49.198 DEBUG:teuthology.orchestra.run.trial068:> sudo yum clean all 2026-02-26T03:11:49.307 INFO:teuthology.orchestra.run.trial012.stdout:41 files removed 2026-02-26T03:11:49.321 DEBUG:teuthology.orchestra.run.trial012:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-osd-crimson ceph-osd-crimson-debuginfo python3-pytest 2026-02-26T03:11:49.347 INFO:teuthology.orchestra.run.trial068.stdout:41 files removed 2026-02-26T03:11:49.361 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-osd-crimson ceph-osd-crimson-debuginfo python3-pytest 2026-02-26T03:11:50.212 INFO:teuthology.orchestra.run.trial068.stdout:ceph packages for x86_64 134 kB/s | 94 kB 00:00 2026-02-26T03:11:50.230 INFO:teuthology.orchestra.run.trial012.stdout:ceph packages for x86_64 123 kB/s | 94 kB 00:00 2026-02-26T03:11:50.745 INFO:teuthology.orchestra.run.trial068.stdout:ceph noarch packages 48 kB/s | 24 kB 00:00 2026-02-26T03:11:50.765 INFO:teuthology.orchestra.run.trial012.stdout:ceph noarch packages 48 kB/s | 24 kB 00:00 2026-02-26T03:11:51.231 INFO:teuthology.orchestra.run.trial068.stdout:ceph source packages 4.5 kB/s | 2.1 kB 00:00 2026-02-26T03:11:51.251 INFO:teuthology.orchestra.run.trial012.stdout:ceph source packages 4.5 kB/s | 2.1 kB 00:00 2026-02-26T03:11:59.555 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.9 MB 00:08 2026-02-26T03:11:59.924 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.9 MB 00:08 2026-02-26T03:12:03.442 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 27 MB 00:03 2026-02-26T03:12:08.265 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - AppStream 3.5 MB/s | 27 MB 00:07 2026-02-26T03:12:13.603 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 8.0 MB 00:02 2026-02-26T03:12:13.804 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - CRB 1.0 MB/s | 8.0 MB 00:07 2026-02-26T03:12:14.545 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - Extras packages 101 kB/s | 20 kB 00:00 2026-02-26T03:12:15.342 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 20 kB 00:00 2026-02-26T03:12:15.460 INFO:teuthology.orchestra.run.trial068.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2026-02-26T03:12:16.096 INFO:teuthology.orchestra.run.trial012.stdout:Extra Packages for Enterprise Linux 29 MB/s | 20 MB 00:00 2026-02-26T03:12:18.609 INFO:teuthology.orchestra.run.trial068.stdout:lab-extras 2.8 MB/s | 50 kB 00:00 2026-02-26T03:12:19.290 INFO:teuthology.orchestra.run.trial012.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-26T03:12:19.836 INFO:teuthology.orchestra.run.trial068.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-26T03:12:19.836 INFO:teuthology.orchestra.run.trial068.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-26T03:12:19.865 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-26T03:12:19.869 INFO:teuthology.orchestra.run.trial068.stdout:======================================================================================= 2026-02-26T03:12:19.870 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repository Size 2026-02-26T03:12:19.870 INFO:teuthology.orchestra.run.trial068.stdout:======================================================================================= 2026-02-26T03:12:19.870 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-26T03:12:19.870 INFO:teuthology.orchestra.run.trial068.stdout: ceph x86_64 2:20.3.0-5312.gabffb617.el9 ceph 6.5 k 2026-02-26T03:12:19.870 INFO:teuthology.orchestra.run.trial068.stdout: ceph-base x86_64 2:20.3.0-5312.gabffb617.el9 ceph 5.1 M 2026-02-26T03:12:19.870 INFO:teuthology.orchestra.run.trial068.stdout: ceph-fuse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 914 k 2026-02-26T03:12:19.870 INFO:teuthology.orchestra.run.trial068.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5312.gabffb617.el9 ceph 149 k 2026-02-26T03:12:19.870 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr x86_64 2:20.3.0-5312.gabffb617.el9 ceph 937 k 2026-02-26T03:12:19.870 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 184 k 2026-02-26T03:12:19.871 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 51 M 2026-02-26T03:12:19.871 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 7.4 M 2026-02-26T03:12:19.871 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-rook noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 50 k 2026-02-26T03:12:19.871 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 163 k 2026-02-26T03:12:19.871 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-classic x86_64 2:20.3.0-5312.gabffb617.el9 ceph 15 M 2026-02-26T03:12:19.871 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-crimson x86_64 2:20.3.0-5312.gabffb617.el9 ceph 87 M 2026-02-26T03:12:19.871 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5312.gabffb617.el9 ceph 429 M 2026-02-26T03:12:19.871 INFO:teuthology.orchestra.run.trial068.stdout: ceph-radosgw x86_64 2:20.3.0-5312.gabffb617.el9 ceph 11 M 2026-02-26T03:12:19.871 INFO:teuthology.orchestra.run.trial068.stdout: ceph-test x86_64 2:20.3.0-5312.gabffb617.el9 ceph 126 M 2026-02-26T03:12:19.871 INFO:teuthology.orchestra.run.trial068.stdout: ceph-volume noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 301 k 2026-02-26T03:12:19.871 INFO:teuthology.orchestra.run.trial068.stdout: cephadm noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 1.0 M 2026-02-26T03:12:19.872 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs-devel x86_64 2:20.3.0-5312.gabffb617.el9 ceph 37 k 2026-02-26T03:12:19.872 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 859 k 2026-02-26T03:12:19.872 INFO:teuthology.orchestra.run.trial068.stdout: librados-devel x86_64 2:20.3.0-5312.gabffb617.el9 ceph 124 k 2026-02-26T03:12:19.872 INFO:teuthology.orchestra.run.trial068.stdout: python3-cephfs x86_64 2:20.3.0-5312.gabffb617.el9 ceph 203 k 2026-02-26T03:12:19.872 INFO:teuthology.orchestra.run.trial068.stdout: python3-pytest noarch 6.2.2-7.el9 appstream 519 k 2026-02-26T03:12:19.872 INFO:teuthology.orchestra.run.trial068.stdout: python3-rados x86_64 2:20.3.0-5312.gabffb617.el9 ceph 324 k 2026-02-26T03:12:19.872 INFO:teuthology.orchestra.run.trial068.stdout: python3-rbd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 305 k 2026-02-26T03:12:19.872 INFO:teuthology.orchestra.run.trial068.stdout: python3-rgw x86_64 2:20.3.0-5312.gabffb617.el9 ceph 100 k 2026-02-26T03:12:19.872 INFO:teuthology.orchestra.run.trial068.stdout: rbd-fuse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 87 k 2026-02-26T03:12:19.872 INFO:teuthology.orchestra.run.trial068.stdout: rbd-mirror x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.6 M 2026-02-26T03:12:19.873 INFO:teuthology.orchestra.run.trial068.stdout: rbd-nbd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 166 k 2026-02-26T03:12:19.873 INFO:teuthology.orchestra.run.trial068.stdout:Upgrading: 2026-02-26T03:12:19.873 INFO:teuthology.orchestra.run.trial068.stdout: librados2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 3.3 M 2026-02-26T03:12:19.873 INFO:teuthology.orchestra.run.trial068.stdout: librbd1 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.7 M 2026-02-26T03:12:19.873 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-26T03:12:19.873 INFO:teuthology.orchestra.run.trial068.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-26T03:12:19.873 INFO:teuthology.orchestra.run.trial068.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-26T03:12:19.873 INFO:teuthology.orchestra.run.trial068.stdout: ceph-common x86_64 2:20.3.0-5312.gabffb617.el9 ceph 21 M 2026-02-26T03:12:19.873 INFO:teuthology.orchestra.run.trial068.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 57 k 2026-02-26T03:12:19.873 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mds x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.0 M 2026-02-26T03:12:19.873 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 297 k 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mon x86_64 2:20.3.0-5312.gabffb617.el9 ceph 4.9 M 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 17 k 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: ceph-selinux x86_64 2:20.3.0-5312.gabffb617.el9 ceph 25 k 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-26T03:12:19.874 INFO:teuthology.orchestra.run.trial068.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 26 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: libcephsqlite x86_64 2:20.3.0-5312.gabffb617.el9 ceph 153 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: libradosstriper1 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 263 k 2026-02-26T03:12:19.875 INFO:teuthology.orchestra.run.trial068.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: librgw2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 5.6 M 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-26T03:12:19.876 INFO:teuthology.orchestra.run.trial068.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: python3-ceph-argparse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 45 k 2026-02-26T03:12:19.877 INFO:teuthology.orchestra.run.trial068.stdout: python3-ceph-common x86_64 2:20.3.0-5312.gabffb617.el9 ceph 185 k 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-iniconfig noarch 1.1.1-7.el9 appstream 17 k 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-26T03:12:19.878 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-26T03:12:19.879 INFO:teuthology.orchestra.run.trial068.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-26T03:12:19.880 INFO:teuthology.orchestra.run.trial068.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-26T03:12:19.880 INFO:teuthology.orchestra.run.trial068.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-26T03:12:19.880 INFO:teuthology.orchestra.run.trial068.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-26T03:12:19.880 INFO:teuthology.orchestra.run.trial068.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-26T03:12:19.880 INFO:teuthology.orchestra.run.trial068.stdout: python3-pluggy noarch 0.13.1-7.el9 appstream 41 k 2026-02-26T03:12:19.880 INFO:teuthology.orchestra.run.trial068.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-26T03:12:19.880 INFO:teuthology.orchestra.run.trial068.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-26T03:12:19.880 INFO:teuthology.orchestra.run.trial068.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-26T03:12:19.880 INFO:teuthology.orchestra.run.trial068.stdout: python3-py noarch 1.10.0-6.el9 appstream 477 k 2026-02-26T03:12:19.880 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-26T03:12:19.880 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-26T03:12:19.881 INFO:teuthology.orchestra.run.trial068.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-26T03:12:19.882 INFO:teuthology.orchestra.run.trial068.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout:Installing weak dependencies: 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout: ceph-debugsource x86_64 2:20.3.0-5312.gabffb617.el9 ceph 12 M 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 22 k 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs-daemon x86_64 2:20.3.0-5312.gabffb617.el9 ceph 37 k 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:12:19.883 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-26T03:12:19.884 INFO:teuthology.orchestra.run.trial068.stdout:======================================================================================= 2026-02-26T03:12:19.884 INFO:teuthology.orchestra.run.trial068.stdout:Install 156 Packages 2026-02-26T03:12:19.884 INFO:teuthology.orchestra.run.trial068.stdout:Upgrade 2 Packages 2026-02-26T03:12:19.884 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:12:19.885 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 863 M 2026-02-26T03:12:19.885 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-26T03:12:20.489 INFO:teuthology.orchestra.run.trial012.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-26T03:12:20.490 INFO:teuthology.orchestra.run.trial012.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-26T03:12:20.519 INFO:teuthology.orchestra.run.trial012.stdout:Dependencies resolved. 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout:======================================================================================= 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: Package Arch Version Repository Size 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout:======================================================================================= 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout:Installing: 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph x86_64 2:20.3.0-5312.gabffb617.el9 ceph 6.5 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-base x86_64 2:20.3.0-5312.gabffb617.el9 ceph 5.1 M 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-fuse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 914 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5312.gabffb617.el9 ceph 149 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr x86_64 2:20.3.0-5312.gabffb617.el9 ceph 937 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 184 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 51 M 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 7.4 M 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-rook noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 50 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-osd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 163 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-osd-classic x86_64 2:20.3.0-5312.gabffb617.el9 ceph 15 M 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-osd-crimson x86_64 2:20.3.0-5312.gabffb617.el9 ceph 87 M 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5312.gabffb617.el9 ceph 429 M 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-radosgw x86_64 2:20.3.0-5312.gabffb617.el9 ceph 11 M 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-test x86_64 2:20.3.0-5312.gabffb617.el9 ceph 126 M 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: ceph-volume noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 301 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: cephadm noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 1.0 M 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: libcephfs-devel x86_64 2:20.3.0-5312.gabffb617.el9 ceph 37 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: libcephfs2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 859 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: librados-devel x86_64 2:20.3.0-5312.gabffb617.el9 ceph 124 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: python3-cephfs x86_64 2:20.3.0-5312.gabffb617.el9 ceph 203 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: python3-pytest noarch 6.2.2-7.el9 appstream 519 k 2026-02-26T03:12:20.523 INFO:teuthology.orchestra.run.trial012.stdout: python3-rados x86_64 2:20.3.0-5312.gabffb617.el9 ceph 324 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: python3-rbd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 305 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: python3-rgw x86_64 2:20.3.0-5312.gabffb617.el9 ceph 100 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: rbd-fuse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 87 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: rbd-mirror x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.6 M 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: rbd-nbd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 166 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout:Upgrading: 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: librados2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 3.3 M 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: librbd1 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.7 M 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout:Installing dependencies: 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: ceph-common x86_64 2:20.3.0-5312.gabffb617.el9 ceph 21 M 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 57 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mds x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.0 M 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 297 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mon x86_64 2:20.3.0-5312.gabffb617.el9 ceph 4.9 M 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 17 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: ceph-selinux x86_64 2:20.3.0-5312.gabffb617.el9 ceph 25 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 26 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libcephsqlite x86_64 2:20.3.0-5312.gabffb617.el9 ceph 153 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libradosstriper1 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 263 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: librgw2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 5.6 M 2026-02-26T03:12:20.524 INFO:teuthology.orchestra.run.trial012.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-ceph-argparse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 45 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-ceph-common x86_64 2:20.3.0-5312.gabffb617.el9 ceph 185 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-iniconfig noarch 1.1.1-7.el9 appstream 17 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-26T03:12:20.525 INFO:teuthology.orchestra.run.trial012.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-pluggy noarch 0.13.1-7.el9 appstream 41 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-py noarch 1.10.0-6.el9 appstream 477 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout:Installing weak dependencies: 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: ceph-debugsource x86_64 2:20.3.0-5312.gabffb617.el9 ceph 12 M 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 22 k 2026-02-26T03:12:20.526 INFO:teuthology.orchestra.run.trial012.stdout: libcephfs-daemon x86_64 2:20.3.0-5312.gabffb617.el9 ceph 37 k 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout:Transaction Summary 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout:======================================================================================= 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout:Install 156 Packages 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout:Upgrade 2 Packages 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout:Total download size: 863 M 2026-02-26T03:12:20.527 INFO:teuthology.orchestra.run.trial012.stdout:Downloading Packages: 2026-02-26T03:12:20.997 INFO:teuthology.orchestra.run.trial068.stdout:(1/158): ceph-20.3.0-5312.gabffb617.el9.x86_64. 25 kB/s | 6.5 kB 00:00 2026-02-26T03:12:21.254 INFO:teuthology.orchestra.run.trial012.stdout:(1/158): ceph-20.3.0-5312.gabffb617.el9.x86_64. 23 kB/s | 6.5 kB 00:00 2026-02-26T03:12:22.979 INFO:teuthology.orchestra.run.trial012.stdout:(2/158): ceph-base-20.3.0-5312.gabffb617.el9.x8 2.5 MB/s | 5.1 MB 00:02 2026-02-26T03:12:23.231 INFO:teuthology.orchestra.run.trial012.stdout:(3/158): ceph-fuse-20.3.0-5312.gabffb617.el9.x8 3.5 MB/s | 914 kB 00:00 2026-02-26T03:12:23.296 INFO:teuthology.orchestra.run.trial012.stdout:(4/158): ceph-immutable-object-cache-20.3.0-531 2.3 MB/s | 149 kB 00:00 2026-02-26T03:12:23.899 INFO:teuthology.orchestra.run.trial012.stdout:(5/158): ceph-mds-20.3.0-5312.gabffb617.el9.x86 3.4 MB/s | 2.0 MB 00:00 2026-02-26T03:12:24.181 INFO:teuthology.orchestra.run.trial012.stdout:(6/158): ceph-mgr-20.3.0-5312.gabffb617.el9.x86 3.2 MB/s | 937 kB 00:00 2026-02-26T03:12:25.831 INFO:teuthology.orchestra.run.trial012.stdout:(7/158): ceph-mon-20.3.0-5312.gabffb617.el9.x86 2.9 MB/s | 4.9 MB 00:01 2026-02-26T03:12:25.902 INFO:teuthology.orchestra.run.trial012.stdout:(8/158): ceph-osd-20.3.0-5312.gabffb617.el9.x86 2.2 MB/s | 163 kB 00:00 2026-02-26T03:12:26.809 INFO:teuthology.orchestra.run.trial068.stdout:(2/158): ceph-base-20.3.0-5312.gabffb617.el9.x8 861 kB/s | 5.1 MB 00:06 2026-02-26T03:12:27.616 INFO:teuthology.orchestra.run.trial068.stdout:(3/158): ceph-fuse-20.3.0-5312.gabffb617.el9.x8 1.1 MB/s | 914 kB 00:00 2026-02-26T03:12:27.754 INFO:teuthology.orchestra.run.trial068.stdout:(4/158): ceph-immutable-object-cache-20.3.0-531 1.1 MB/s | 149 kB 00:00 2026-02-26T03:12:27.790 INFO:teuthology.orchestra.run.trial012.stdout:(9/158): ceph-debugsource-20.3.0-5312.gabffb617 1.8 MB/s | 12 MB 00:06 2026-02-26T03:12:29.260 INFO:teuthology.orchestra.run.trial068.stdout:(5/158): ceph-mds-20.3.0-5312.gabffb617.el9.x86 1.3 MB/s | 2.0 MB 00:01 2026-02-26T03:12:29.860 INFO:teuthology.orchestra.run.trial068.stdout:(6/158): ceph-mgr-20.3.0-5312.gabffb617.el9.x86 1.5 MB/s | 937 kB 00:00 2026-02-26T03:12:31.800 INFO:teuthology.orchestra.run.trial012.stdout:(10/158): ceph-osd-classic-20.3.0-5312.gabffb61 2.5 MB/s | 15 MB 00:05 2026-02-26T03:12:32.698 INFO:teuthology.orchestra.run.trial068.stdout:(7/158): ceph-mon-20.3.0-5312.gabffb617.el9.x86 1.7 MB/s | 4.9 MB 00:02 2026-02-26T03:12:32.843 INFO:teuthology.orchestra.run.trial068.stdout:(8/158): ceph-osd-20.3.0-5312.gabffb617.el9.x86 1.1 MB/s | 163 kB 00:00 2026-02-26T03:12:33.193 INFO:teuthology.orchestra.run.trial012.stdout:(11/158): ceph-common-20.3.0-5312.gabffb617.el9 1.7 MB/s | 21 MB 00:12 2026-02-26T03:12:35.882 INFO:teuthology.orchestra.run.trial068.stdout:(9/158): ceph-debugsource-20.3.0-5312.gabffb617 828 kB/s | 12 MB 00:14 2026-02-26T03:12:42.520 INFO:teuthology.orchestra.run.trial068.stdout:(10/158): ceph-common-20.3.0-5312.gabffb617.el9 978 kB/s | 21 MB 00:21 2026-02-26T03:12:43.233 INFO:teuthology.orchestra.run.trial068.stdout:(11/158): ceph-osd-classic-20.3.0-5312.gabffb61 1.4 MB/s | 15 MB 00:10 2026-02-26T03:12:45.277 INFO:teuthology.orchestra.run.trial012.stdout:(12/158): ceph-radosgw-20.3.0-5312.gabffb617.el 947 kB/s | 11 MB 00:12 2026-02-26T03:12:45.341 INFO:teuthology.orchestra.run.trial012.stdout:(13/158): ceph-selinux-20.3.0-5312.gabffb617.el 393 kB/s | 25 kB 00:00 2026-02-26T03:12:47.776 INFO:teuthology.orchestra.run.trial068.stdout:(12/158): ceph-radosgw-20.3.0-5312.gabffb617.el 2.5 MB/s | 11 MB 00:04 2026-02-26T03:12:47.844 INFO:teuthology.orchestra.run.trial068.stdout:(13/158): ceph-selinux-20.3.0-5312.gabffb617.el 374 kB/s | 25 kB 00:00 2026-02-26T03:13:40.599 INFO:teuthology.orchestra.run.trial012.stdout:(14/158): ceph-osd-crimson-20.3.0-5312.gabffb61 1.2 MB/s | 87 MB 01:12 2026-02-26T03:13:40.666 INFO:teuthology.orchestra.run.trial012.stdout:(15/158): libcephfs-daemon-20.3.0-5312.gabffb61 550 kB/s | 37 kB 00:00 2026-02-26T03:13:40.735 INFO:teuthology.orchestra.run.trial012.stdout:(16/158): libcephfs-devel-20.3.0-5312.gabffb617 542 kB/s | 37 kB 00:00 2026-02-26T03:13:40.806 INFO:teuthology.orchestra.run.trial012.stdout:(17/158): libcephfs-proxy2-20.3.0-5312.gabffb61 363 kB/s | 26 kB 00:00 2026-02-26T03:13:41.872 INFO:teuthology.orchestra.run.trial012.stdout:(18/158): libcephfs2-20.3.0-5312.gabffb617.el9. 806 kB/s | 859 kB 00:01 2026-02-26T03:13:42.082 INFO:teuthology.orchestra.run.trial012.stdout:(19/158): libcephsqlite-20.3.0-5312.gabffb617.e 730 kB/s | 153 kB 00:00 2026-02-26T03:13:42.292 INFO:teuthology.orchestra.run.trial012.stdout:(20/158): librados-devel-20.3.0-5312.gabffb617. 587 kB/s | 124 kB 00:00 2026-02-26T03:13:42.608 INFO:teuthology.orchestra.run.trial012.stdout:(21/158): libradosstriper1-20.3.0-5312.gabffb61 834 kB/s | 263 kB 00:00 2026-02-26T03:13:46.141 INFO:teuthology.orchestra.run.trial068.stdout:(14/158): ceph-osd-crimson-20.3.0-5312.gabffb61 1.2 MB/s | 87 MB 01:10 2026-02-26T03:13:46.209 INFO:teuthology.orchestra.run.trial068.stdout:(15/158): libcephfs-daemon-20.3.0-5312.gabffb61 542 kB/s | 37 kB 00:00 2026-02-26T03:13:46.280 INFO:teuthology.orchestra.run.trial068.stdout:(16/158): libcephfs-devel-20.3.0-5312.gabffb617 530 kB/s | 37 kB 00:00 2026-02-26T03:13:46.336 INFO:teuthology.orchestra.run.trial068.stdout:(17/158): libcephfs-proxy2-20.3.0-5312.gabffb61 456 kB/s | 26 kB 00:00 2026-02-26T03:13:47.679 INFO:teuthology.orchestra.run.trial068.stdout:(18/158): libcephfs2-20.3.0-5312.gabffb617.el9. 640 kB/s | 859 kB 00:01 2026-02-26T03:13:47.890 INFO:teuthology.orchestra.run.trial068.stdout:(19/158): libcephsqlite-20.3.0-5312.gabffb617.e 728 kB/s | 153 kB 00:00 2026-02-26T03:13:48.096 INFO:teuthology.orchestra.run.trial068.stdout:(20/158): librados-devel-20.3.0-5312.gabffb617. 600 kB/s | 124 kB 00:00 2026-02-26T03:13:48.449 INFO:teuthology.orchestra.run.trial068.stdout:(21/158): libradosstriper1-20.3.0-5312.gabffb61 746 kB/s | 263 kB 00:00 2026-02-26T03:13:51.440 INFO:teuthology.orchestra.run.trial012.stdout:(22/158): librgw2-20.3.0-5312.gabffb617.el9.x86 652 kB/s | 5.6 MB 00:08 2026-02-26T03:13:51.506 INFO:teuthology.orchestra.run.trial012.stdout:(23/158): python3-ceph-argparse-20.3.0-5312.gab 676 kB/s | 45 kB 00:00 2026-02-26T03:13:51.683 INFO:teuthology.orchestra.run.trial012.stdout:(24/158): python3-ceph-common-20.3.0-5312.gabff 1.0 MB/s | 185 kB 00:00 2026-02-26T03:13:51.922 INFO:teuthology.orchestra.run.trial012.stdout:(25/158): python3-cephfs-20.3.0-5312.gabffb617. 853 kB/s | 203 kB 00:00 2026-02-26T03:13:52.228 INFO:teuthology.orchestra.run.trial012.stdout:(26/158): python3-rados-20.3.0-5312.gabffb617.e 1.0 MB/s | 324 kB 00:00 2026-02-26T03:13:52.534 INFO:teuthology.orchestra.run.trial012.stdout:(27/158): python3-rbd-20.3.0-5312.gabffb617.el9 1.0 MB/s | 305 kB 00:00 2026-02-26T03:13:52.655 INFO:teuthology.orchestra.run.trial012.stdout:(28/158): python3-rgw-20.3.0-5312.gabffb617.el9 829 kB/s | 100 kB 00:00 2026-02-26T03:13:52.776 INFO:teuthology.orchestra.run.trial012.stdout:(29/158): rbd-fuse-20.3.0-5312.gabffb617.el9.x8 714 kB/s | 87 kB 00:00 2026-02-26T03:13:53.747 INFO:teuthology.orchestra.run.trial068.stdout:(22/158): librgw2-20.3.0-5312.gabffb617.el9.x86 1.1 MB/s | 5.6 MB 00:05 2026-02-26T03:13:53.818 INFO:teuthology.orchestra.run.trial068.stdout:(23/158): python3-ceph-argparse-20.3.0-5312.gab 632 kB/s | 45 kB 00:00 2026-02-26T03:13:54.023 INFO:teuthology.orchestra.run.trial068.stdout:(24/158): python3-ceph-common-20.3.0-5312.gabff 905 kB/s | 185 kB 00:00 2026-02-26T03:13:54.229 INFO:teuthology.orchestra.run.trial068.stdout:(25/158): python3-cephfs-20.3.0-5312.gabffb617. 989 kB/s | 203 kB 00:00 2026-02-26T03:13:54.572 INFO:teuthology.orchestra.run.trial068.stdout:(26/158): python3-rados-20.3.0-5312.gabffb617.e 945 kB/s | 324 kB 00:00 2026-02-26T03:13:54.847 INFO:teuthology.orchestra.run.trial068.stdout:(27/158): python3-rbd-20.3.0-5312.gabffb617.el9 1.1 MB/s | 305 kB 00:00 2026-02-26T03:13:54.986 INFO:teuthology.orchestra.run.trial068.stdout:(28/158): python3-rgw-20.3.0-5312.gabffb617.el9 721 kB/s | 100 kB 00:00 2026-02-26T03:13:55.118 INFO:teuthology.orchestra.run.trial068.stdout:(29/158): rbd-fuse-20.3.0-5312.gabffb617.el9.x8 658 kB/s | 87 kB 00:00 2026-02-26T03:13:55.792 INFO:teuthology.orchestra.run.trial012.stdout:(30/158): rbd-mirror-20.3.0-5312.gabffb617.el9. 880 kB/s | 2.6 MB 00:03 2026-02-26T03:13:56.189 INFO:teuthology.orchestra.run.trial012.stdout:(31/158): rbd-nbd-20.3.0-5312.gabffb617.el9.x86 418 kB/s | 166 kB 00:00 2026-02-26T03:13:56.329 INFO:teuthology.orchestra.run.trial012.stdout:(32/158): ceph-grafana-dashboards-20.3.0-5312.g 410 kB/s | 57 kB 00:00 2026-02-26T03:13:56.834 INFO:teuthology.orchestra.run.trial012.stdout:(33/158): ceph-mgr-cephadm-20.3.0-5312.gabffb61 364 kB/s | 184 kB 00:00 2026-02-26T03:13:57.095 INFO:teuthology.orchestra.run.trial068.stdout:(30/158): rbd-mirror-20.3.0-5312.gabffb617.el9. 1.3 MB/s | 2.6 MB 00:01 2026-02-26T03:13:57.229 INFO:teuthology.orchestra.run.trial068.stdout:(31/158): rbd-nbd-20.3.0-5312.gabffb617.el9.x86 1.2 MB/s | 166 kB 00:00 2026-02-26T03:13:57.297 INFO:teuthology.orchestra.run.trial068.stdout:(32/158): ceph-grafana-dashboards-20.3.0-5312.g 844 kB/s | 57 kB 00:00 2026-02-26T03:13:57.436 INFO:teuthology.orchestra.run.trial068.stdout:(33/158): ceph-mgr-cephadm-20.3.0-5312.gabffb61 1.3 MB/s | 184 kB 00:00 2026-02-26T03:14:42.153 INFO:teuthology.orchestra.run.trial012.stdout:(34/158): ceph-test-20.3.0-5312.gabffb617.el9.x 1.1 MB/s | 126 MB 01:56 2026-02-26T03:14:48.946 INFO:teuthology.orchestra.run.trial068.stdout:(34/158): ceph-mgr-dashboard-20.3.0-5312.gabffb 1.0 MB/s | 51 MB 00:51 2026-02-26T03:14:49.356 INFO:teuthology.orchestra.run.trial012.stdout:(35/158): ceph-mgr-diskprediction-local-20.3.0- 1.0 MB/s | 7.4 MB 00:07 2026-02-26T03:14:49.427 INFO:teuthology.orchestra.run.trial012.stdout:(36/158): ceph-mgr-k8sevents-20.3.0-5312.gabffb 317 kB/s | 22 kB 00:00 2026-02-26T03:14:49.849 INFO:teuthology.orchestra.run.trial012.stdout:(37/158): ceph-mgr-modules-core-20.3.0-5312.gab 703 kB/s | 297 kB 00:00 2026-02-26T03:14:49.921 INFO:teuthology.orchestra.run.trial012.stdout:(38/158): ceph-mgr-rook-20.3.0-5312.gabffb617.e 700 kB/s | 50 kB 00:00 2026-02-26T03:14:49.990 INFO:teuthology.orchestra.run.trial012.stdout:(39/158): ceph-prometheus-alerts-20.3.0-5312.ga 251 kB/s | 17 kB 00:00 2026-02-26T03:14:50.396 INFO:teuthology.orchestra.run.trial012.stdout:(40/158): ceph-volume-20.3.0-5312.gabffb617.el9 743 kB/s | 301 kB 00:00 2026-02-26T03:14:51.914 INFO:teuthology.orchestra.run.trial012.stdout:(41/158): cephadm-20.3.0-5312.gabffb617.el9.noa 685 kB/s | 1.0 MB 00:01 2026-02-26T03:14:52.563 INFO:teuthology.orchestra.run.trial012.stdout:(42/158): cryptsetup-2.8.1-3.el9.x86_64.rpm 541 kB/s | 351 kB 00:00 2026-02-26T03:14:52.809 INFO:teuthology.orchestra.run.trial012.stdout:(43/158): hwloc-libs-2.4.1-6.el9.x86_64.rpm 8.5 MB/s | 2.1 MB 00:00 2026-02-26T03:14:52.892 INFO:teuthology.orchestra.run.trial012.stdout:(44/158): ledmon-libs-1.1.0-3.el9.x86_64.rpm 484 kB/s | 40 kB 00:00 2026-02-26T03:14:52.976 INFO:teuthology.orchestra.run.trial012.stdout:(45/158): libatomic-11.5.0-14.el9.x86_64.rpm 274 kB/s | 23 kB 00:00 2026-02-26T03:14:53.059 INFO:teuthology.orchestra.run.trial012.stdout:(46/158): libconfig-1.7.2-9.el9.x86_64.rpm 866 kB/s | 72 kB 00:00 2026-02-26T03:14:53.147 INFO:teuthology.orchestra.run.trial012.stdout:(47/158): libgfortran-11.5.0-14.el9.x86_64.rpm 8.9 MB/s | 794 kB 00:00 2026-02-26T03:14:53.231 INFO:teuthology.orchestra.run.trial012.stdout:(48/158): libquadmath-11.5.0-14.el9.x86_64.rpm 2.1 MB/s | 184 kB 00:00 2026-02-26T03:14:53.316 INFO:teuthology.orchestra.run.trial012.stdout:(49/158): lksctp-tools-1.0.19-2.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2026-02-26T03:14:53.400 INFO:teuthology.orchestra.run.trial012.stdout:(50/158): mailcap-2.1.49-5.el9.noarch.rpm 394 kB/s | 33 kB 00:00 2026-02-26T03:14:53.488 INFO:teuthology.orchestra.run.trial012.stdout:(51/158): nvme-cli-2.16-1.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2026-02-26T03:14:53.572 INFO:teuthology.orchestra.run.trial012.stdout:(52/158): pciutils-3.7.0-7.el9.x86_64.rpm 1.1 MB/s | 93 kB 00:00 2026-02-26T03:14:53.657 INFO:teuthology.orchestra.run.trial012.stdout:(53/158): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 253 kB 00:00 2026-02-26T03:14:53.746 INFO:teuthology.orchestra.run.trial012.stdout:(54/158): python3-cryptography-36.0.1-5.el9.x86 14 MB/s | 1.2 MB 00:00 2026-02-26T03:14:53.834 INFO:teuthology.orchestra.run.trial012.stdout:(55/158): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2026-02-26T03:14:53.920 INFO:teuthology.orchestra.run.trial012.stdout:(56/158): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 135 kB 00:00 2026-02-26T03:14:54.006 INFO:teuthology.orchestra.run.trial012.stdout:(57/158): python3-pyparsing-2.4.7-9.el9.noarch. 1.7 MB/s | 150 kB 00:00 2026-02-26T03:14:54.098 INFO:teuthology.orchestra.run.trial012.stdout:(58/158): python3-requests-2.25.1-10.el9.noarch 1.3 MB/s | 126 kB 00:00 2026-02-26T03:14:54.190 INFO:teuthology.orchestra.run.trial012.stdout:(59/158): python3-urllib3-1.26.5-7.el9.noarch.r 2.3 MB/s | 218 kB 00:00 2026-02-26T03:14:54.274 INFO:teuthology.orchestra.run.trial012.stdout:(60/158): unzip-6.0-59.el9.x86_64.rpm 2.1 MB/s | 182 kB 00:00 2026-02-26T03:14:54.359 INFO:teuthology.orchestra.run.trial012.stdout:(61/158): zip-3.0-35.el9.x86_64.rpm 3.1 MB/s | 266 kB 00:00 2026-02-26T03:14:54.759 INFO:teuthology.orchestra.run.trial012.stdout:(62/158): boost-program-options-1.75.0-13.el9.x 259 kB/s | 104 kB 00:00 2026-02-26T03:14:54.852 INFO:teuthology.orchestra.run.trial012.stdout:(63/158): flexiblas-3.0.4-9.el9.x86_64.rpm 320 kB/s | 30 kB 00:00 2026-02-26T03:14:55.324 INFO:teuthology.orchestra.run.trial012.stdout:(64/158): flexiblas-netlib-3.0.4-9.el9.x86_64.r 6.3 MB/s | 3.0 MB 00:00 2026-02-26T03:14:55.403 INFO:teuthology.orchestra.run.trial012.stdout:(65/158): flexiblas-openblas-openmp-3.0.4-9.el9 188 kB/s | 15 kB 00:00 2026-02-26T03:14:55.491 INFO:teuthology.orchestra.run.trial012.stdout:(66/158): libasan8-15.2.1-7.el9.x86_64.rpm 5.9 MB/s | 525 kB 00:00 2026-02-26T03:14:55.571 INFO:teuthology.orchestra.run.trial012.stdout:(67/158): libnbd-1.20.3-4.el9.x86_64.rpm 2.0 MB/s | 164 kB 00:00 2026-02-26T03:14:55.652 INFO:teuthology.orchestra.run.trial012.stdout:(68/158): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2026-02-26T03:14:55.731 INFO:teuthology.orchestra.run.trial012.stdout:(69/158): librabbitmq-0.11.0-7.el9.x86_64.rpm 570 kB/s | 45 kB 00:00 2026-02-26T03:14:55.821 INFO:teuthology.orchestra.run.trial012.stdout:(70/158): librdkafka-1.6.1-102.el9.x86_64.rpm 7.3 MB/s | 662 kB 00:00 2026-02-26T03:14:55.903 INFO:teuthology.orchestra.run.trial012.stdout:(71/158): libstoragemgmt-1.10.1-1.el9.x86_64.rp 2.9 MB/s | 246 kB 00:00 2026-02-26T03:14:55.984 INFO:teuthology.orchestra.run.trial012.stdout:(72/158): libubsan-11.5.0-14.el9.x86_64.rpm 2.1 MB/s | 176 kB 00:00 2026-02-26T03:14:56.066 INFO:teuthology.orchestra.run.trial012.stdout:(73/158): libxslt-1.1.34-12.el9.x86_64.rpm 2.8 MB/s | 233 kB 00:00 2026-02-26T03:14:56.148 INFO:teuthology.orchestra.run.trial012.stdout:(74/158): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.5 MB/s | 292 kB 00:00 2026-02-26T03:14:56.229 INFO:teuthology.orchestra.run.trial012.stdout:(75/158): lua-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 188 kB 00:00 2026-02-26T03:14:56.308 INFO:teuthology.orchestra.run.trial012.stdout:(76/158): openblas-0.3.29-1.el9.x86_64.rpm 533 kB/s | 42 kB 00:00 2026-02-26T03:14:56.516 INFO:teuthology.orchestra.run.trial012.stdout:(77/158): openblas-openmp-0.3.29-1.el9.x86_64.r 25 MB/s | 5.3 MB 00:00 2026-02-26T03:14:56.603 INFO:teuthology.orchestra.run.trial012.stdout:(78/158): protobuf-3.14.0-17.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2026-02-26T03:14:56.790 INFO:teuthology.orchestra.run.trial012.stdout:(79/158): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2026-02-26T03:14:56.871 INFO:teuthology.orchestra.run.trial012.stdout:(80/158): python3-devel-3.9.25-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2026-02-26T03:14:56.950 INFO:teuthology.orchestra.run.trial012.stdout:(81/158): python3-iniconfig-1.1.1-7.el9.noarch. 221 kB/s | 17 kB 00:00 2026-02-26T03:14:57.032 INFO:teuthology.orchestra.run.trial012.stdout:(82/158): python3-jinja2-2.11.3-8.el9.noarch.rp 3.0 MB/s | 249 kB 00:00 2026-02-26T03:14:57.112 INFO:teuthology.orchestra.run.trial012.stdout:(83/158): python3-jmespath-1.0.1-1.el9.noarch.r 596 kB/s | 48 kB 00:00 2026-02-26T03:14:57.193 INFO:teuthology.orchestra.run.trial012.stdout:(84/158): python3-libstoragemgmt-1.10.1-1.el9.x 2.1 MB/s | 177 kB 00:00 2026-02-26T03:14:57.282 INFO:teuthology.orchestra.run.trial012.stdout:(85/158): python3-lxml-4.6.5-3.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2026-02-26T03:14:57.361 INFO:teuthology.orchestra.run.trial012.stdout:(86/158): python3-markupsafe-1.1.1-12.el9.x86_6 435 kB/s | 35 kB 00:00 2026-02-26T03:14:57.628 INFO:teuthology.orchestra.run.trial012.stdout:(87/158): python3-numpy-1.23.5-2.el9.x86_64.rpm 23 MB/s | 6.1 MB 00:00 2026-02-26T03:14:57.711 INFO:teuthology.orchestra.run.trial012.stdout:(88/158): python3-numpy-f2py-1.23.5-2.el9.x86_6 5.2 MB/s | 442 kB 00:00 2026-02-26T03:14:57.791 INFO:teuthology.orchestra.run.trial012.stdout:(89/158): python3-packaging-20.9-5.el9.noarch.r 960 kB/s | 77 kB 00:00 2026-02-26T03:14:57.871 INFO:teuthology.orchestra.run.trial012.stdout:(90/158): python3-pluggy-0.13.1-7.el9.noarch.rp 521 kB/s | 41 kB 00:00 2026-02-26T03:14:57.951 INFO:teuthology.orchestra.run.trial012.stdout:(91/158): python3-protobuf-3.14.0-17.el9.noarch 3.2 MB/s | 267 kB 00:00 2026-02-26T03:14:58.034 INFO:teuthology.orchestra.run.trial012.stdout:(92/158): python3-py-1.10.0-6.el9.noarch.rpm 5.6 MB/s | 477 kB 00:00 2026-02-26T03:14:58.116 INFO:teuthology.orchestra.run.trial012.stdout:(93/158): python3-pyasn1-0.4.8-7.el9.noarch.rpm 1.9 MB/s | 157 kB 00:00 2026-02-26T03:14:58.198 INFO:teuthology.orchestra.run.trial012.stdout:(94/158): python3-pyasn1-modules-0.4.8-7.el9.no 3.3 MB/s | 277 kB 00:00 2026-02-26T03:14:58.287 INFO:teuthology.orchestra.run.trial012.stdout:(95/158): python3-pytest-6.2.2-7.el9.noarch.rpm 5.7 MB/s | 519 kB 00:00 2026-02-26T03:14:58.367 INFO:teuthology.orchestra.run.trial012.stdout:(96/158): python3-requests-oauthlib-1.3.0-12.el 668 kB/s | 54 kB 00:00 2026-02-26T03:14:58.987 INFO:teuthology.orchestra.run.trial012.stdout:(97/158): python3-scipy-1.9.3-2.el9.x86_64.rpm 31 MB/s | 19 MB 00:00 2026-02-26T03:14:59.067 INFO:teuthology.orchestra.run.trial012.stdout:(98/158): python3-toml-0.10.2-6.el9.noarch.rpm 520 kB/s | 42 kB 00:00 2026-02-26T03:14:59.150 INFO:teuthology.orchestra.run.trial012.stdout:(99/158): qatlib-25.08.0-2.el9.x86_64.rpm 2.8 MB/s | 240 kB 00:00 2026-02-26T03:14:59.229 INFO:teuthology.orchestra.run.trial012.stdout:(100/158): qatlib-service-25.08.0-2.el9.x86_64. 469 kB/s | 37 kB 00:00 2026-02-26T03:14:59.309 INFO:teuthology.orchestra.run.trial012.stdout:(101/158): qatzip-libs-1.3.1-1.el9.x86_64.rpm 836 kB/s | 66 kB 00:00 2026-02-26T03:14:59.393 INFO:teuthology.orchestra.run.trial012.stdout:(102/158): socat-1.7.4.1-8.el9.x86_64.rpm 3.5 MB/s | 303 kB 00:00 2026-02-26T03:14:59.474 INFO:teuthology.orchestra.run.trial012.stdout:(103/158): xmlsec1-1.2.29-13.el9.x86_64.rpm 2.3 MB/s | 189 kB 00:00 2026-02-26T03:14:59.555 INFO:teuthology.orchestra.run.trial012.stdout:(104/158): xmlsec1-openssl-1.2.29-13.el9.x86_64 1.1 MB/s | 90 kB 00:00 2026-02-26T03:14:59.634 INFO:teuthology.orchestra.run.trial012.stdout:(105/158): xmlstarlet-1.6.1-20.el9.x86_64.rpm 802 kB/s | 64 kB 00:00 2026-02-26T03:14:59.692 INFO:teuthology.orchestra.run.trial012.stdout:(106/158): lua-devel-5.4.4-4.el9.x86_64.rpm 383 kB/s | 22 kB 00:00 2026-02-26T03:14:59.835 INFO:teuthology.orchestra.run.trial012.stdout:(107/158): protobuf-compiler-3.14.0-17.el9.x86_ 5.9 MB/s | 862 kB 00:00 2026-02-26T03:15:00.197 INFO:teuthology.orchestra.run.trial068.stdout:(35/158): ceph-mgr-diskprediction-local-20.3.0- 672 kB/s | 7.4 MB 00:11 2026-02-26T03:15:00.240 INFO:teuthology.orchestra.run.trial012.stdout:(108/158): abseil-cpp-20211102.0-4.el9.x86_64.r 1.3 MB/s | 551 kB 00:00 2026-02-26T03:15:00.266 INFO:teuthology.orchestra.run.trial068.stdout:(36/158): ceph-mgr-k8sevents-20.3.0-5312.gabffb 323 kB/s | 22 kB 00:00 2026-02-26T03:15:00.315 INFO:teuthology.orchestra.run.trial012.stdout:(109/158): gperftools-libs-2.9.1-3.el9.x86_64.r 4.0 MB/s | 308 kB 00:00 2026-02-26T03:15:00.368 INFO:teuthology.orchestra.run.trial012.stdout:(110/158): grpc-data-1.46.7-10.el9.noarch.rpm 375 kB/s | 19 kB 00:00 2026-02-26T03:15:00.635 INFO:teuthology.orchestra.run.trial012.stdout:(111/158): libarrow-9.0.0-15.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2026-02-26T03:15:00.679 INFO:teuthology.orchestra.run.trial068.stdout:(37/158): ceph-mgr-modules-core-20.3.0-5312.gab 719 kB/s | 297 kB 00:00 2026-02-26T03:15:00.686 INFO:teuthology.orchestra.run.trial012.stdout:(112/158): libarrow-doc-9.0.0-15.el9.noarch.rpm 487 kB/s | 25 kB 00:00 2026-02-26T03:15:00.744 INFO:teuthology.orchestra.run.trial012.stdout:(113/158): liboath-2.6.12-1.el9.x86_64.rpm 850 kB/s | 49 kB 00:00 2026-02-26T03:15:00.750 INFO:teuthology.orchestra.run.trial068.stdout:(38/158): ceph-mgr-rook-20.3.0-5312.gabffb617.e 713 kB/s | 50 kB 00:00 2026-02-26T03:15:00.795 INFO:teuthology.orchestra.run.trial012.stdout:(114/158): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2026-02-26T03:15:00.819 INFO:teuthology.orchestra.run.trial068.stdout:(39/158): ceph-prometheus-alerts-20.3.0-5312.ga 249 kB/s | 17 kB 00:00 2026-02-26T03:15:00.847 INFO:teuthology.orchestra.run.trial012.stdout:(115/158): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2026-02-26T03:15:00.905 INFO:teuthology.orchestra.run.trial012.stdout:(116/158): parquet-libs-9.0.0-15.el9.x86_64.rpm 14 MB/s | 838 kB 00:00 2026-02-26T03:15:00.961 INFO:teuthology.orchestra.run.trial012.stdout:(117/158): python3-asyncssh-2.13.2-5.el9.noarch 9.7 MB/s | 548 kB 00:00 2026-02-26T03:15:01.012 INFO:teuthology.orchestra.run.trial012.stdout:(118/158): python3-autocommand-2.2.2-8.el9.noar 574 kB/s | 29 kB 00:00 2026-02-26T03:15:01.063 INFO:teuthology.orchestra.run.trial012.stdout:(119/158): python3-backports-tarfile-1.2.0-1.el 1.2 MB/s | 60 kB 00:00 2026-02-26T03:15:01.114 INFO:teuthology.orchestra.run.trial012.stdout:(120/158): python3-bcrypt-3.2.2-1.el9.x86_64.rp 853 kB/s | 43 kB 00:00 2026-02-26T03:15:01.165 INFO:teuthology.orchestra.run.trial012.stdout:(121/158): python3-cachetools-4.2.4-1.el9.noarc 633 kB/s | 32 kB 00:00 2026-02-26T03:15:01.216 INFO:teuthology.orchestra.run.trial012.stdout:(122/158): python3-certifi-2023.05.07-4.el9.noa 276 kB/s | 14 kB 00:00 2026-02-26T03:15:01.234 INFO:teuthology.orchestra.run.trial068.stdout:(40/158): ceph-volume-20.3.0-5312.gabffb617.el9 727 kB/s | 301 kB 00:00 2026-02-26T03:15:01.318 INFO:teuthology.orchestra.run.trial012.stdout:(123/158): python3-cheroot-10.0.1-4.el9.noarch. 1.7 MB/s | 173 kB 00:00 2026-02-26T03:15:01.372 INFO:teuthology.orchestra.run.trial012.stdout:(124/158): python3-cherrypy-18.6.1-2.el9.noarch 6.5 MB/s | 358 kB 00:00 2026-02-26T03:15:01.424 INFO:teuthology.orchestra.run.trial012.stdout:(125/158): python3-google-auth-2.45.0-1.el9.noa 4.7 MB/s | 254 kB 00:00 2026-02-26T03:15:01.493 INFO:teuthology.orchestra.run.trial012.stdout:(126/158): python3-grpcio-1.46.7-10.el9.x86_64. 30 MB/s | 2.0 MB 00:00 2026-02-26T03:15:01.545 INFO:teuthology.orchestra.run.trial012.stdout:(127/158): python3-grpcio-tools-1.46.7-10.el9.x 2.7 MB/s | 144 kB 00:00 2026-02-26T03:15:01.597 INFO:teuthology.orchestra.run.trial012.stdout:(128/158): python3-influxdb-5.3.1-1.el9.noarch. 2.6 MB/s | 139 kB 00:00 2026-02-26T03:15:01.648 INFO:teuthology.orchestra.run.trial012.stdout:(129/158): python3-isodate-0.6.1-3.el9.noarch.r 1.1 MB/s | 56 kB 00:00 2026-02-26T03:15:01.699 INFO:teuthology.orchestra.run.trial012.stdout:(130/158): python3-jaraco-8.2.1-3.el9.noarch.rp 210 kB/s | 11 kB 00:00 2026-02-26T03:15:01.750 INFO:teuthology.orchestra.run.trial012.stdout:(131/158): python3-jaraco-classes-3.2.1-5.el9.n 348 kB/s | 18 kB 00:00 2026-02-26T03:15:01.800 INFO:teuthology.orchestra.run.trial012.stdout:(132/158): python3-jaraco-collections-3.0.0-8.e 455 kB/s | 23 kB 00:00 2026-02-26T03:15:01.851 INFO:teuthology.orchestra.run.trial012.stdout:(133/158): python3-jaraco-context-6.0.1-3.el9.n 387 kB/s | 20 kB 00:00 2026-02-26T03:15:01.902 INFO:teuthology.orchestra.run.trial012.stdout:(134/158): python3-jaraco-functools-3.5.0-2.el9 382 kB/s | 19 kB 00:00 2026-02-26T03:15:01.953 INFO:teuthology.orchestra.run.trial012.stdout:(135/158): python3-jaraco-text-4.0.0-2.el9.noar 520 kB/s | 26 kB 00:00 2026-02-26T03:15:02.013 INFO:teuthology.orchestra.run.trial012.stdout:(136/158): python3-kubernetes-26.1.0-3.el9.noar 17 MB/s | 1.0 MB 00:00 2026-02-26T03:15:02.064 INFO:teuthology.orchestra.run.trial012.stdout:(137/158): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2026-02-26T03:15:02.201 INFO:teuthology.orchestra.run.trial012.stdout:(138/158): python3-msgpack-1.0.3-2.el9.x86_64.r 631 kB/s | 86 kB 00:00 2026-02-26T03:15:02.252 INFO:teuthology.orchestra.run.trial012.stdout:(139/158): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2026-02-26T03:15:02.304 INFO:teuthology.orchestra.run.trial012.stdout:(140/158): python3-portend-3.1.0-2.el9.noarch.r 318 kB/s | 16 kB 00:00 2026-02-26T03:15:02.331 INFO:teuthology.orchestra.run.trial068.stdout:(41/158): cephadm-20.3.0-5312.gabffb617.el9.noa 948 kB/s | 1.0 MB 00:01 2026-02-26T03:15:02.356 INFO:teuthology.orchestra.run.trial012.stdout:(141/158): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2026-02-26T03:15:02.406 INFO:teuthology.orchestra.run.trial012.stdout:(142/158): python3-repoze-lru-0.7-16.el9.noarch 604 kB/s | 31 kB 00:00 2026-02-26T03:15:02.508 INFO:teuthology.orchestra.run.trial012.stdout:(143/158): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2026-02-26T03:15:02.574 INFO:teuthology.orchestra.run.trial012.stdout:(144/158): python3-rsa-4.9-2.el9.noarch.rpm 896 kB/s | 59 kB 00:00 2026-02-26T03:15:02.658 INFO:teuthology.orchestra.run.trial012.stdout:(145/158): python3-saml-1.16.0-1.el9.noarch.rpm 1.4 MB/s | 125 kB 00:00 2026-02-26T03:15:02.718 INFO:teuthology.orchestra.run.trial012.stdout:(146/158): python3-tempora-5.0.0-2.el9.noarch.r 599 kB/s | 36 kB 00:00 2026-02-26T03:15:02.793 INFO:teuthology.orchestra.run.trial012.stdout:(147/158): python3-typing-extensions-4.15.0-1.e 1.1 MB/s | 86 kB 00:00 2026-02-26T03:15:02.867 INFO:teuthology.orchestra.run.trial012.stdout:(148/158): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2026-02-26T03:15:02.930 INFO:teuthology.orchestra.run.trial012.stdout:(149/158): python3-xmlsec-1.3.13-1.el9.x86_64.r 767 kB/s | 48 kB 00:00 2026-02-26T03:15:02.986 INFO:teuthology.orchestra.run.trial012.stdout:(150/158): python3-xmltodict-0.12.0-15.el9.noar 396 kB/s | 22 kB 00:00 2026-02-26T03:15:03.041 INFO:teuthology.orchestra.run.trial012.stdout:(151/158): python3-zc-lockfile-2.0-10.el9.noarc 362 kB/s | 20 kB 00:00 2026-02-26T03:15:03.137 INFO:teuthology.orchestra.run.trial068.stdout:(42/158): cryptsetup-2.8.1-3.el9.x86_64.rpm 436 kB/s | 351 kB 00:00 2026-02-26T03:15:03.144 INFO:teuthology.orchestra.run.trial012.stdout:(152/158): re2-20211101-20.el9.x86_64.rpm 1.8 MB/s | 191 kB 00:00 2026-02-26T03:15:03.597 INFO:teuthology.orchestra.run.trial012.stdout:(153/158): thrift-0.15.0-4.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2026-02-26T03:15:03.676 INFO:teuthology.orchestra.run.trial012.stdout:(154/158): yaml-cpp-0.6.3-5.el9.x86_64.rpm 1.5 MB/s | 122 kB 00:00 2026-02-26T03:15:04.080 INFO:teuthology.orchestra.run.trial068.stdout:(43/158): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.2 MB/s | 2.1 MB 00:00 2026-02-26T03:15:04.163 INFO:teuthology.orchestra.run.trial068.stdout:(44/158): ledmon-libs-1.1.0-3.el9.x86_64.rpm 485 kB/s | 40 kB 00:00 2026-02-26T03:15:04.246 INFO:teuthology.orchestra.run.trial068.stdout:(45/158): libatomic-11.5.0-14.el9.x86_64.rpm 275 kB/s | 23 kB 00:00 2026-02-26T03:15:04.330 INFO:teuthology.orchestra.run.trial068.stdout:(46/158): libconfig-1.7.2-9.el9.x86_64.rpm 869 kB/s | 72 kB 00:00 2026-02-26T03:15:04.396 INFO:teuthology.orchestra.run.trial012.stdout:(155/158): ceph-mgr-dashboard-20.3.0-5312.gabff 767 kB/s | 51 MB 01:07 2026-02-26T03:15:04.416 INFO:teuthology.orchestra.run.trial068.stdout:(47/158): libgfortran-11.5.0-14.el9.x86_64.rpm 9.0 MB/s | 794 kB 00:00 2026-02-26T03:15:04.500 INFO:teuthology.orchestra.run.trial068.stdout:(48/158): libquadmath-11.5.0-14.el9.x86_64.rpm 2.2 MB/s | 184 kB 00:00 2026-02-26T03:15:04.583 INFO:teuthology.orchestra.run.trial068.stdout:(49/158): lksctp-tools-1.0.19-2.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2026-02-26T03:15:04.666 INFO:teuthology.orchestra.run.trial068.stdout:(50/158): mailcap-2.1.49-5.el9.noarch.rpm 398 kB/s | 33 kB 00:00 2026-02-26T03:15:04.755 INFO:teuthology.orchestra.run.trial068.stdout:(51/158): nvme-cli-2.16-1.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2026-02-26T03:15:04.838 INFO:teuthology.orchestra.run.trial068.stdout:(52/158): pciutils-3.7.0-7.el9.x86_64.rpm 1.1 MB/s | 93 kB 00:00 2026-02-26T03:15:04.922 INFO:teuthology.orchestra.run.trial068.stdout:(53/158): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 253 kB 00:00 2026-02-26T03:15:05.011 INFO:teuthology.orchestra.run.trial068.stdout:(54/158): python3-cryptography-36.0.1-5.el9.x86 14 MB/s | 1.2 MB 00:00 2026-02-26T03:15:05.095 INFO:teuthology.orchestra.run.trial068.stdout:(55/158): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2026-02-26T03:15:05.180 INFO:teuthology.orchestra.run.trial068.stdout:(56/158): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2026-02-26T03:15:05.264 INFO:teuthology.orchestra.run.trial068.stdout:(57/158): python3-pyparsing-2.4.7-9.el9.noarch. 1.8 MB/s | 150 kB 00:00 2026-02-26T03:15:05.347 INFO:teuthology.orchestra.run.trial068.stdout:(58/158): python3-requests-2.25.1-10.el9.noarch 1.5 MB/s | 126 kB 00:00 2026-02-26T03:15:05.432 INFO:teuthology.orchestra.run.trial068.stdout:(59/158): python3-urllib3-1.26.5-7.el9.noarch.r 2.5 MB/s | 218 kB 00:00 2026-02-26T03:15:05.516 INFO:teuthology.orchestra.run.trial068.stdout:(60/158): unzip-6.0-59.el9.x86_64.rpm 2.1 MB/s | 182 kB 00:00 2026-02-26T03:15:05.600 INFO:teuthology.orchestra.run.trial068.stdout:(61/158): zip-3.0-35.el9.x86_64.rpm 3.1 MB/s | 266 kB 00:00 2026-02-26T03:15:06.001 INFO:teuthology.orchestra.run.trial068.stdout:(62/158): boost-program-options-1.75.0-13.el9.x 259 kB/s | 104 kB 00:00 2026-02-26T03:15:06.093 INFO:teuthology.orchestra.run.trial068.stdout:(63/158): flexiblas-3.0.4-9.el9.x86_64.rpm 323 kB/s | 30 kB 00:00 2026-02-26T03:15:06.559 INFO:teuthology.orchestra.run.trial068.stdout:(64/158): flexiblas-netlib-3.0.4-9.el9.x86_64.r 6.4 MB/s | 3.0 MB 00:00 2026-02-26T03:15:06.637 INFO:teuthology.orchestra.run.trial068.stdout:(65/158): flexiblas-openblas-openmp-3.0.4-9.el9 190 kB/s | 15 kB 00:00 2026-02-26T03:15:06.723 INFO:teuthology.orchestra.run.trial068.stdout:(66/158): libasan8-15.2.1-7.el9.x86_64.rpm 5.9 MB/s | 525 kB 00:00 2026-02-26T03:15:06.804 INFO:teuthology.orchestra.run.trial068.stdout:(67/158): libnbd-1.20.3-4.el9.x86_64.rpm 2.0 MB/s | 164 kB 00:00 2026-02-26T03:15:06.884 INFO:teuthology.orchestra.run.trial068.stdout:(68/158): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.0 MB/s | 160 kB 00:00 2026-02-26T03:15:06.962 INFO:teuthology.orchestra.run.trial068.stdout:(69/158): librabbitmq-0.11.0-7.el9.x86_64.rpm 577 kB/s | 45 kB 00:00 2026-02-26T03:15:07.051 INFO:teuthology.orchestra.run.trial068.stdout:(70/158): librdkafka-1.6.1-102.el9.x86_64.rpm 7.3 MB/s | 662 kB 00:00 2026-02-26T03:15:07.132 INFO:teuthology.orchestra.run.trial068.stdout:(71/158): libstoragemgmt-1.10.1-1.el9.x86_64.rp 3.0 MB/s | 246 kB 00:00 2026-02-26T03:15:07.212 INFO:teuthology.orchestra.run.trial068.stdout:(72/158): libubsan-11.5.0-14.el9.x86_64.rpm 2.1 MB/s | 176 kB 00:00 2026-02-26T03:15:07.293 INFO:teuthology.orchestra.run.trial068.stdout:(73/158): libxslt-1.1.34-12.el9.x86_64.rpm 2.8 MB/s | 233 kB 00:00 2026-02-26T03:15:07.375 INFO:teuthology.orchestra.run.trial068.stdout:(74/158): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.5 MB/s | 292 kB 00:00 2026-02-26T03:15:07.455 INFO:teuthology.orchestra.run.trial068.stdout:(75/158): lua-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 188 kB 00:00 2026-02-26T03:15:07.534 INFO:teuthology.orchestra.run.trial068.stdout:(76/158): openblas-0.3.29-1.el9.x86_64.rpm 538 kB/s | 42 kB 00:00 2026-02-26T03:15:07.740 INFO:teuthology.orchestra.run.trial068.stdout:(77/158): openblas-openmp-0.3.29-1.el9.x86_64.r 26 MB/s | 5.3 MB 00:00 2026-02-26T03:15:07.826 INFO:teuthology.orchestra.run.trial068.stdout:(78/158): protobuf-3.14.0-17.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2026-02-26T03:15:07.891 INFO:teuthology.orchestra.run.trial012.stdout:(156/158): librbd1-20.3.0-5312.gabffb617.el9.x8 788 kB/s | 2.7 MB 00:03 2026-02-26T03:15:08.010 INFO:teuthology.orchestra.run.trial068.stdout:(79/158): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2026-02-26T03:15:08.092 INFO:teuthology.orchestra.run.trial068.stdout:(80/158): python3-devel-3.9.25-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2026-02-26T03:15:08.170 INFO:teuthology.orchestra.run.trial068.stdout:(81/158): python3-iniconfig-1.1.1-7.el9.noarch. 222 kB/s | 17 kB 00:00 2026-02-26T03:15:08.251 INFO:teuthology.orchestra.run.trial068.stdout:(82/158): python3-jinja2-2.11.3-8.el9.noarch.rp 3.0 MB/s | 249 kB 00:00 2026-02-26T03:15:08.331 INFO:teuthology.orchestra.run.trial068.stdout:(83/158): python3-jmespath-1.0.1-1.el9.noarch.r 597 kB/s | 48 kB 00:00 2026-02-26T03:15:08.410 INFO:teuthology.orchestra.run.trial068.stdout:(84/158): python3-libstoragemgmt-1.10.1-1.el9.x 2.2 MB/s | 177 kB 00:00 2026-02-26T03:15:08.469 INFO:teuthology.orchestra.run.trial012.stdout:(157/158): librados2-20.3.0-5312.gabffb617.el9. 702 kB/s | 3.3 MB 00:04 2026-02-26T03:15:08.499 INFO:teuthology.orchestra.run.trial068.stdout:(85/158): python3-lxml-4.6.5-3.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2026-02-26T03:15:08.577 INFO:teuthology.orchestra.run.trial068.stdout:(86/158): python3-markupsafe-1.1.1-12.el9.x86_6 442 kB/s | 35 kB 00:00 2026-02-26T03:15:08.764 INFO:teuthology.orchestra.run.trial068.stdout:(87/158): python3-numpy-1.23.5-2.el9.x86_64.rpm 33 MB/s | 6.1 MB 00:00 2026-02-26T03:15:08.845 INFO:teuthology.orchestra.run.trial068.stdout:(88/158): python3-numpy-f2py-1.23.5-2.el9.x86_6 5.3 MB/s | 442 kB 00:00 2026-02-26T03:15:08.925 INFO:teuthology.orchestra.run.trial068.stdout:(89/158): python3-packaging-20.9-5.el9.noarch.r 968 kB/s | 77 kB 00:00 2026-02-26T03:15:09.004 INFO:teuthology.orchestra.run.trial068.stdout:(90/158): python3-pluggy-0.13.1-7.el9.noarch.rp 526 kB/s | 41 kB 00:00 2026-02-26T03:15:09.084 INFO:teuthology.orchestra.run.trial068.stdout:(91/158): python3-protobuf-3.14.0-17.el9.noarch 3.3 MB/s | 267 kB 00:00 2026-02-26T03:15:09.166 INFO:teuthology.orchestra.run.trial068.stdout:(92/158): python3-py-1.10.0-6.el9.noarch.rpm 5.7 MB/s | 477 kB 00:00 2026-02-26T03:15:09.247 INFO:teuthology.orchestra.run.trial068.stdout:(93/158): python3-pyasn1-0.4.8-7.el9.noarch.rpm 1.9 MB/s | 157 kB 00:00 2026-02-26T03:15:09.328 INFO:teuthology.orchestra.run.trial068.stdout:(94/158): python3-pyasn1-modules-0.4.8-7.el9.no 3.3 MB/s | 277 kB 00:00 2026-02-26T03:15:09.416 INFO:teuthology.orchestra.run.trial068.stdout:(95/158): python3-pytest-6.2.2-7.el9.noarch.rpm 5.8 MB/s | 519 kB 00:00 2026-02-26T03:15:09.496 INFO:teuthology.orchestra.run.trial068.stdout:(96/158): python3-requests-oauthlib-1.3.0-12.el 678 kB/s | 54 kB 00:00 2026-02-26T03:15:10.112 INFO:teuthology.orchestra.run.trial068.stdout:(97/158): python3-scipy-1.9.3-2.el9.x86_64.rpm 31 MB/s | 19 MB 00:00 2026-02-26T03:15:10.191 INFO:teuthology.orchestra.run.trial068.stdout:(98/158): python3-toml-0.10.2-6.el9.noarch.rpm 526 kB/s | 42 kB 00:00 2026-02-26T03:15:10.273 INFO:teuthology.orchestra.run.trial068.stdout:(99/158): qatlib-25.08.0-2.el9.x86_64.rpm 2.9 MB/s | 240 kB 00:00 2026-02-26T03:15:10.351 INFO:teuthology.orchestra.run.trial068.stdout:(100/158): qatlib-service-25.08.0-2.el9.x86_64. 476 kB/s | 37 kB 00:00 2026-02-26T03:15:10.430 INFO:teuthology.orchestra.run.trial068.stdout:(101/158): qatzip-libs-1.3.1-1.el9.x86_64.rpm 845 kB/s | 66 kB 00:00 2026-02-26T03:15:10.513 INFO:teuthology.orchestra.run.trial068.stdout:(102/158): socat-1.7.4.1-8.el9.x86_64.rpm 3.6 MB/s | 303 kB 00:00 2026-02-26T03:15:10.594 INFO:teuthology.orchestra.run.trial068.stdout:(103/158): xmlsec1-1.2.29-13.el9.x86_64.rpm 2.3 MB/s | 189 kB 00:00 2026-02-26T03:15:10.673 INFO:teuthology.orchestra.run.trial068.stdout:(104/158): xmlsec1-openssl-1.2.29-13.el9.x86_64 1.1 MB/s | 90 kB 00:00 2026-02-26T03:15:10.751 INFO:teuthology.orchestra.run.trial068.stdout:(105/158): xmlstarlet-1.6.1-20.el9.x86_64.rpm 814 kB/s | 64 kB 00:00 2026-02-26T03:15:10.762 INFO:teuthology.orchestra.run.trial068.stdout:(106/158): lua-devel-5.4.4-4.el9.x86_64.rpm 2.1 MB/s | 22 kB 00:00 2026-02-26T03:15:10.781 INFO:teuthology.orchestra.run.trial068.stdout:(107/158): protobuf-compiler-3.14.0-17.el9.x86_ 45 MB/s | 862 kB 00:00 2026-02-26T03:15:11.246 INFO:teuthology.orchestra.run.trial068.stdout:(108/158): abseil-cpp-20211102.0-4.el9.x86_64.r 1.2 MB/s | 551 kB 00:00 2026-02-26T03:15:11.331 INFO:teuthology.orchestra.run.trial068.stdout:(109/158): gperftools-libs-2.9.1-3.el9.x86_64.r 3.6 MB/s | 308 kB 00:00 2026-02-26T03:15:11.393 INFO:teuthology.orchestra.run.trial068.stdout:(110/158): grpc-data-1.46.7-10.el9.noarch.rpm 314 kB/s | 19 kB 00:00 2026-02-26T03:15:11.655 INFO:teuthology.orchestra.run.trial068.stdout:(111/158): libarrow-9.0.0-15.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2026-02-26T03:15:11.717 INFO:teuthology.orchestra.run.trial068.stdout:(112/158): libarrow-doc-9.0.0-15.el9.noarch.rpm 401 kB/s | 25 kB 00:00 2026-02-26T03:15:11.779 INFO:teuthology.orchestra.run.trial068.stdout:(113/158): liboath-2.6.12-1.el9.x86_64.rpm 792 kB/s | 49 kB 00:00 2026-02-26T03:15:11.841 INFO:teuthology.orchestra.run.trial068.stdout:(114/158): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2026-02-26T03:15:11.905 INFO:teuthology.orchestra.run.trial068.stdout:(115/158): luarocks-3.9.2-5.el9.noarch.rpm 2.3 MB/s | 151 kB 00:00 2026-02-26T03:15:11.984 INFO:teuthology.orchestra.run.trial068.stdout:(116/158): parquet-libs-9.0.0-15.el9.x86_64.rpm 10 MB/s | 838 kB 00:00 2026-02-26T03:15:12.050 INFO:teuthology.orchestra.run.trial068.stdout:(117/158): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2026-02-26T03:15:12.113 INFO:teuthology.orchestra.run.trial068.stdout:(118/158): python3-autocommand-2.2.2-8.el9.noar 472 kB/s | 29 kB 00:00 2026-02-26T03:15:12.175 INFO:teuthology.orchestra.run.trial068.stdout:(119/158): python3-backports-tarfile-1.2.0-1.el 961 kB/s | 60 kB 00:00 2026-02-26T03:15:12.238 INFO:teuthology.orchestra.run.trial068.stdout:(120/158): python3-bcrypt-3.2.2-1.el9.x86_64.rp 698 kB/s | 43 kB 00:00 2026-02-26T03:15:12.300 INFO:teuthology.orchestra.run.trial068.stdout:(121/158): python3-cachetools-4.2.4-1.el9.noarc 521 kB/s | 32 kB 00:00 2026-02-26T03:15:12.361 INFO:teuthology.orchestra.run.trial068.stdout:(122/158): python3-certifi-2023.05.07-4.el9.noa 228 kB/s | 14 kB 00:00 2026-02-26T03:15:12.425 INFO:teuthology.orchestra.run.trial068.stdout:(123/158): python3-cheroot-10.0.1-4.el9.noarch. 2.7 MB/s | 173 kB 00:00 2026-02-26T03:15:12.494 INFO:teuthology.orchestra.run.trial068.stdout:(124/158): python3-cherrypy-18.6.1-2.el9.noarch 5.1 MB/s | 358 kB 00:00 2026-02-26T03:15:12.559 INFO:teuthology.orchestra.run.trial068.stdout:(125/158): python3-google-auth-2.45.0-1.el9.noa 3.8 MB/s | 254 kB 00:00 2026-02-26T03:15:12.664 INFO:teuthology.orchestra.run.trial068.stdout:(126/158): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2026-02-26T03:15:12.728 INFO:teuthology.orchestra.run.trial068.stdout:(127/158): python3-grpcio-tools-1.46.7-10.el9.x 2.2 MB/s | 144 kB 00:00 2026-02-26T03:15:12.792 INFO:teuthology.orchestra.run.trial068.stdout:(128/158): python3-influxdb-5.3.1-1.el9.noarch. 2.1 MB/s | 139 kB 00:00 2026-02-26T03:15:12.855 INFO:teuthology.orchestra.run.trial068.stdout:(129/158): python3-isodate-0.6.1-3.el9.noarch.r 895 kB/s | 56 kB 00:00 2026-02-26T03:15:12.917 INFO:teuthology.orchestra.run.trial068.stdout:(130/158): python3-jaraco-8.2.1-3.el9.noarch.rp 172 kB/s | 11 kB 00:00 2026-02-26T03:15:12.979 INFO:teuthology.orchestra.run.trial068.stdout:(131/158): python3-jaraco-classes-3.2.1-5.el9.n 285 kB/s | 18 kB 00:00 2026-02-26T03:15:13.041 INFO:teuthology.orchestra.run.trial068.stdout:(132/158): python3-jaraco-collections-3.0.0-8.e 373 kB/s | 23 kB 00:00 2026-02-26T03:15:13.103 INFO:teuthology.orchestra.run.trial068.stdout:(133/158): python3-jaraco-context-6.0.1-3.el9.n 316 kB/s | 20 kB 00:00 2026-02-26T03:15:13.165 INFO:teuthology.orchestra.run.trial068.stdout:(134/158): python3-jaraco-functools-3.5.0-2.el9 314 kB/s | 19 kB 00:00 2026-02-26T03:15:13.227 INFO:teuthology.orchestra.run.trial068.stdout:(135/158): python3-jaraco-text-4.0.0-2.el9.noar 424 kB/s | 26 kB 00:00 2026-02-26T03:15:13.300 INFO:teuthology.orchestra.run.trial068.stdout:(136/158): python3-kubernetes-26.1.0-3.el9.noar 14 MB/s | 1.0 MB 00:00 2026-02-26T03:15:13.363 INFO:teuthology.orchestra.run.trial068.stdout:(137/158): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2026-02-26T03:15:13.426 INFO:teuthology.orchestra.run.trial068.stdout:(138/158): python3-msgpack-1.0.3-2.el9.x86_64.r 1.3 MB/s | 86 kB 00:00 2026-02-26T03:15:13.488 INFO:teuthology.orchestra.run.trial068.stdout:(139/158): python3-natsort-7.1.1-5.el9.noarch.r 920 kB/s | 58 kB 00:00 2026-02-26T03:15:13.550 INFO:teuthology.orchestra.run.trial068.stdout:(140/158): python3-portend-3.1.0-2.el9.noarch.r 265 kB/s | 16 kB 00:00 2026-02-26T03:15:13.613 INFO:teuthology.orchestra.run.trial068.stdout:(141/158): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.4 MB/s | 90 kB 00:00 2026-02-26T03:15:13.675 INFO:teuthology.orchestra.run.trial068.stdout:(142/158): python3-repoze-lru-0.7-16.el9.noarch 494 kB/s | 31 kB 00:00 2026-02-26T03:15:13.739 INFO:teuthology.orchestra.run.trial068.stdout:(143/158): python3-routes-2.5.1-5.el9.noarch.rp 2.9 MB/s | 188 kB 00:00 2026-02-26T03:15:13.803 INFO:teuthology.orchestra.run.trial068.stdout:(144/158): python3-rsa-4.9-2.el9.noarch.rpm 932 kB/s | 59 kB 00:00 2026-02-26T03:15:13.866 INFO:teuthology.orchestra.run.trial068.stdout:(145/158): python3-saml-1.16.0-1.el9.noarch.rpm 1.9 MB/s | 125 kB 00:00 2026-02-26T03:15:13.928 INFO:teuthology.orchestra.run.trial068.stdout:(146/158): python3-tempora-5.0.0-2.el9.noarch.r 575 kB/s | 36 kB 00:00 2026-02-26T03:15:13.991 INFO:teuthology.orchestra.run.trial068.stdout:(147/158): python3-typing-extensions-4.15.0-1.e 1.3 MB/s | 86 kB 00:00 2026-02-26T03:15:14.054 INFO:teuthology.orchestra.run.trial068.stdout:(148/158): python3-websocket-client-1.2.3-2.el9 1.4 MB/s | 90 kB 00:00 2026-02-26T03:15:14.116 INFO:teuthology.orchestra.run.trial068.stdout:(149/158): python3-xmlsec-1.3.13-1.el9.x86_64.r 772 kB/s | 48 kB 00:00 2026-02-26T03:15:14.178 INFO:teuthology.orchestra.run.trial068.stdout:(150/158): python3-xmltodict-0.12.0-15.el9.noar 357 kB/s | 22 kB 00:00 2026-02-26T03:15:14.241 INFO:teuthology.orchestra.run.trial068.stdout:(151/158): python3-zc-lockfile-2.0-10.el9.noarc 322 kB/s | 20 kB 00:00 2026-02-26T03:15:14.303 INFO:teuthology.orchestra.run.trial068.stdout:(152/158): re2-20211101-20.el9.x86_64.rpm 3.0 MB/s | 191 kB 00:00 2026-02-26T03:15:14.399 INFO:teuthology.orchestra.run.trial068.stdout:(153/158): thrift-0.15.0-4.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2026-02-26T03:15:14.462 INFO:teuthology.orchestra.run.trial068.stdout:(154/158): yaml-cpp-0.6.3-5.el9.x86_64.rpm 1.9 MB/s | 122 kB 00:00 2026-02-26T03:15:15.331 INFO:teuthology.orchestra.run.trial068.stdout:(155/158): ceph-test-20.3.0-5312.gabffb617.el9. 874 kB/s | 126 MB 02:27 2026-02-26T03:15:18.377 INFO:teuthology.orchestra.run.trial068.stdout:(156/158): librbd1-20.3.0-5312.gabffb617.el9.x8 904 kB/s | 2.7 MB 00:03 2026-02-26T03:15:19.778 INFO:teuthology.orchestra.run.trial068.stdout:(157/158): librados2-20.3.0-5312.gabffb617.el9. 633 kB/s | 3.3 MB 00:05 2026-02-26T03:21:04.209 INFO:teuthology.orchestra.run.trial012.stdout:(158/158): ceph-osd-crimson-debuginfo-20.3.0-53 857 kB/s | 429 MB 08:32 2026-02-26T03:21:04.216 INFO:teuthology.orchestra.run.trial012.stdout:-------------------------------------------------------------------------------- 2026-02-26T03:21:04.216 INFO:teuthology.orchestra.run.trial012.stdout:Total 1.6 MB/s | 863 MB 08:43 2026-02-26T03:21:04.756 INFO:teuthology.orchestra.run.trial012.stdout:Running transaction check 2026-02-26T03:21:04.808 INFO:teuthology.orchestra.run.trial012.stdout:Transaction check succeeded. 2026-02-26T03:21:04.808 INFO:teuthology.orchestra.run.trial012.stdout:Running transaction test 2026-02-26T03:21:07.553 INFO:teuthology.orchestra.run.trial012.stdout:Transaction test succeeded. 2026-02-26T03:21:07.553 INFO:teuthology.orchestra.run.trial012.stdout:Running transaction 2026-02-26T03:21:10.367 INFO:teuthology.orchestra.run.trial012.stdout: Preparing : 1/1 2026-02-26T03:21:10.370 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/160 2026-02-26T03:21:10.378 INFO:teuthology.orchestra.run.trial012.stdout: Installing : liboath-2.6.12-1.el9.x86_64 2/160 2026-02-26T03:21:10.390 INFO:teuthology.orchestra.run.trial012.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/160 2026-02-26T03:21:10.531 INFO:teuthology.orchestra.run.trial012.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2026-02-26T03:21:10.533 INFO:teuthology.orchestra.run.trial012.stdout: Upgrading : librados2-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:21:10.578 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: librados2-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:21:10.579 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 6/160 2026-02-26T03:21:10.590 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 6/160 2026-02-26T03:21:10.592 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86 7/160 2026-02-26T03:21:10.592 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 8/160 2026-02-26T03:21:10.619 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 8/160 2026-02-26T03:21:10.624 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 9/160 2026-02-26T03:21:10.632 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 10/160 2026-02-26T03:21:10.635 INFO:teuthology.orchestra.run.trial012.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 11/160 2026-02-26T03:21:10.637 INFO:teuthology.orchestra.run.trial012.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 12/160 2026-02-26T03:21:10.639 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 13/160 2026-02-26T03:21:10.644 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 14/160 2026-02-26T03:21:10.757 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 15/160 2026-02-26T03:21:10.758 INFO:teuthology.orchestra.run.trial012.stdout: Upgrading : librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:21:10.799 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:21:10.805 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 17/160 2026-02-26T03:21:10.813 INFO:teuthology.orchestra.run.trial012.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 18/160 2026-02-26T03:21:10.814 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 19/160 2026-02-26T03:21:10.836 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 19/160 2026-02-26T03:21:10.837 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 20/160 2026-02-26T03:21:10.849 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 20/160 2026-02-26T03:21:10.853 INFO:teuthology.orchestra.run.trial012.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 21/160 2026-02-26T03:21:10.884 INFO:teuthology.orchestra.run.trial012.stdout: Installing : re2-1:20211101-20.el9.x86_64 22/160 2026-02-26T03:21:10.889 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 23/160 2026-02-26T03:21:10.893 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 24/160 2026-02-26T03:21:10.914 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 25/160 2026-02-26T03:21:10.919 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 26/160 2026-02-26T03:21:10.923 INFO:teuthology.orchestra.run.trial012.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2026-02-26T03:21:10.927 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 28/160 2026-02-26T03:21:10.928 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 29/160 2026-02-26T03:21:10.940 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 29/160 2026-02-26T03:21:10.946 INFO:teuthology.orchestra.run.trial012.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 30/160 2026-02-26T03:21:10.969 INFO:teuthology.orchestra.run.trial012.stdout: Installing : unzip-6.0-59.el9.x86_64 31/160 2026-02-26T03:21:10.982 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 32/160 2026-02-26T03:21:10.987 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 33/160 2026-02-26T03:21:10.990 INFO:teuthology.orchestra.run.trial012.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 34/160 2026-02-26T03:21:10.996 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 35/160 2026-02-26T03:21:10.998 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 36/160 2026-02-26T03:21:10.999 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 37/160 2026-02-26T03:21:11.009 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 38/160 2026-02-26T03:21:11.044 INFO:teuthology.orchestra.run.trial012.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 39/160 2026-02-26T03:21:11.048 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-ceph-common-2:20.3.0-5312.gabffb617.el9. 40/160 2026-02-26T03:21:11.059 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-ceph-argparse-2:20.3.0-5312.gabffb617.el 41/160 2026-02-26T03:21:11.070 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_6 42/160 2026-02-26T03:21:11.076 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 43/160 2026-02-26T03:21:11.100 INFO:teuthology.orchestra.run.trial012.stdout: Installing : zip-3.0-35.el9.x86_64 44/160 2026-02-26T03:21:11.103 INFO:teuthology.orchestra.run.trial012.stdout: Installing : luarocks-3.9.2-5.el9.noarch 45/160 2026-02-26T03:21:11.154 INFO:teuthology.orchestra.run.trial012.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 46/160 2026-02-26T03:21:11.167 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 47/160 2026-02-26T03:21:11.174 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-rsa-4.9-2.el9.noarch 48/160 2026-02-26T03:21:11.177 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/160 2026-02-26T03:21:11.179 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/160 2026-02-26T03:21:11.183 INFO:teuthology.orchestra.run.trial012.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 51/160 2026-02-26T03:21:11.200 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 52/160 2026-02-26T03:21:11.205 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 53/160 2026-02-26T03:21:11.208 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/160 2026-02-26T03:21:11.215 INFO:teuthology.orchestra.run.trial012.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 55/160 2026-02-26T03:21:11.223 INFO:teuthology.orchestra.run.trial012.stdout: Installing : librados-devel-2:20.3.0-5312.gabffb617.el9.x86_6 56/160 2026-02-26T03:21:11.234 INFO:teuthology.orchestra.run.trial012.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 57/160 2026-02-26T03:21:11.240 INFO:teuthology.orchestra.run.trial012.stdout: Installing : thrift-0.15.0-4.el9.x86_64 58/160 2026-02-26T03:21:11.243 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 59/160 2026-02-26T03:21:11.257 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 60/160 2026-02-26T03:21:11.262 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 61/160 2026-02-26T03:21:11.267 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 62/160 2026-02-26T03:21:11.278 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 63/160 2026-02-26T03:21:11.287 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 64/160 2026-02-26T03:21:11.293 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 65/160 2026-02-26T03:21:11.313 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 66/160 2026-02-26T03:21:11.324 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 67/160 2026-02-26T03:21:11.340 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 68/160 2026-02-26T03:21:11.344 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 69/160 2026-02-26T03:21:11.352 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 70/160 2026-02-26T03:21:11.392 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 71/160 2026-02-26T03:21:11.712 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 72/160 2026-02-26T03:21:11.721 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 73/160 2026-02-26T03:21:11.725 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 74/160 2026-02-26T03:21:11.731 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 75/160 2026-02-26T03:21:11.737 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 76/160 2026-02-26T03:21:11.740 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 77/160 2026-02-26T03:21:11.745 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/160 2026-02-26T03:21:11.751 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 79/160 2026-02-26T03:21:11.754 INFO:teuthology.orchestra.run.trial012.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 80/160 2026-02-26T03:21:11.755 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 81/160 2026-02-26T03:21:11.783 INFO:teuthology.orchestra.run.trial012.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 82/160 2026-02-26T03:21:11.827 INFO:teuthology.orchestra.run.trial012.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 83/160 2026-02-26T03:21:11.837 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 84/160 2026-02-26T03:21:11.882 INFO:teuthology.orchestra.run.trial012.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 85/160 2026-02-26T03:21:11.911 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-py-1.10.0-6.el9.noarch 86/160 2026-02-26T03:21:11.922 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 87/160 2026-02-26T03:21:11.928 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 88/160 2026-02-26T03:21:11.934 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 89/160 2026-02-26T03:21:11.941 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 90/160 2026-02-26T03:21:11.945 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 91/160 2026-02-26T03:21:11.979 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 92/160 2026-02-26T03:21:12.202 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 93/160 2026-02-26T03:21:12.226 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 94/160 2026-02-26T03:21:12.229 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 95/160 2026-02-26T03:21:12.288 INFO:teuthology.orchestra.run.trial012.stdout: Installing : openblas-0.3.29-1.el9.x86_64 96/160 2026-02-26T03:21:12.290 INFO:teuthology.orchestra.run.trial012.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 97/160 2026-02-26T03:21:12.312 INFO:teuthology.orchestra.run.trial012.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 98/160 2026-02-26T03:21:12.638 INFO:teuthology.orchestra.run.trial012.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 99/160 2026-02-26T03:21:12.710 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 100/160 2026-02-26T03:21:13.383 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 101/160 2026-02-26T03:21:13.397 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 102/160 2026-02-26T03:21:13.403 INFO:teuthology.orchestra.run.trial012.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 103/160 2026-02-26T03:21:13.648 INFO:teuthology.orchestra.run.trial012.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 104/160 2026-02-26T03:21:13.650 INFO:teuthology.orchestra.run.trial012.stdout: Installing : librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 105/160 2026-02-26T03:21:13.667 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 105/160 2026-02-26T03:21:13.669 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 106/160 2026-02-26T03:21:14.610 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:21:14.612 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:21:14.629 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:21:14.639 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 108/160 2026-02-26T03:21:14.646 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-packaging-20.9-5.el9.noarch 109/160 2026-02-26T03:21:14.659 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-ply-3.11-14.el9.noarch 110/160 2026-02-26T03:21:14.675 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 111/160 2026-02-26T03:21:14.750 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 112/160 2026-02-26T03:21:14.759 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 113/160 2026-02-26T03:21:14.782 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 114/160 2026-02-26T03:21:14.812 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 115/160 2026-02-26T03:21:14.862 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 116/160 2026-02-26T03:21:14.869 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 117/160 2026-02-26T03:21:14.872 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 118/160 2026-02-26T03:21:14.876 INFO:teuthology.orchestra.run.trial012.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 119/160 2026-02-26T03:21:14.878 INFO:teuthology.orchestra.run.trial012.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 120/160 2026-02-26T03:21:14.879 INFO:teuthology.orchestra.run.trial012.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 121/160 2026-02-26T03:21:14.895 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 121/160 2026-02-26T03:21:15.009 INFO:teuthology.orchestra.run.trial012.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 122/160 2026-02-26T03:21:15.011 INFO:teuthology.orchestra.run.trial012.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 123/160 2026-02-26T03:21:15.294 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 123/160 2026-02-26T03:21:15.295 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-26T03:21:15.295 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:15.552 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 124/160 2026-02-26T03:21:15.590 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 124/160 2026-02-26T03:21:15.591 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-26T03:21:15.591 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-26T03:21:15.591 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:15.594 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 125/160 2026-02-26T03:21:20.295 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 125/160 2026-02-26T03:21:20.295 INFO:teuthology.orchestra.run.trial012.stdout:skipping the directory /sys 2026-02-26T03:21:20.295 INFO:teuthology.orchestra.run.trial012.stdout:skipping the directory /proc 2026-02-26T03:21:20.295 INFO:teuthology.orchestra.run.trial012.stdout:skipping the directory /mnt 2026-02-26T03:21:20.295 INFO:teuthology.orchestra.run.trial012.stdout:skipping the directory /var/tmp 2026-02-26T03:21:20.296 INFO:teuthology.orchestra.run.trial012.stdout:skipping the directory /home 2026-02-26T03:21:20.296 INFO:teuthology.orchestra.run.trial012.stdout:skipping the directory /root 2026-02-26T03:21:20.296 INFO:teuthology.orchestra.run.trial012.stdout:skipping the directory /tmp 2026-02-26T03:21:20.296 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:20.385 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 126/160 2026-02-26T03:21:20.407 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 126/160 2026-02-26T03:21:20.407 INFO:teuthology.orchestra.run.trial012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:21:20.407 INFO:teuthology.orchestra.run.trial012.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-26T03:21:20.407 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-26T03:21:20.407 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-26T03:21:20.407 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:20.612 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 127/160 2026-02-26T03:21:20.633 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 127/160 2026-02-26T03:21:20.633 INFO:teuthology.orchestra.run.trial012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:21:20.633 INFO:teuthology.orchestra.run.trial012.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-26T03:21:20.633 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-26T03:21:20.634 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-26T03:21:20.634 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:20.640 INFO:teuthology.orchestra.run.trial012.stdout: Installing : mailcap-2.1.49-5.el9.noarch 128/160 2026-02-26T03:21:20.642 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 129/160 2026-02-26T03:21:20.660 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:21:20.660 INFO:teuthology.orchestra.run.trial012.stdout:Creating group 'qat' with GID 992. 2026-02-26T03:21:20.660 INFO:teuthology.orchestra.run.trial012.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-26T03:21:20.660 INFO:teuthology.orchestra.run.trial012.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-26T03:21:20.660 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:20.668 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:21:20.690 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:21:20.690 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-26T03:21:20.690 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:20.723 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 131/160 2026-02-26T03:21:21.311 INFO:teuthology.orchestra.run.trial012.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 132/160 2026-02-26T03:21:26.637 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 133/160 2026-02-26T03:21:26.699 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 134/160 2026-02-26T03:21:26.701 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 135/160 2026-02-26T03:21:26.715 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 135/160 2026-02-26T03:21:26.715 INFO:teuthology.orchestra.run.trial012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:21:26.715 INFO:teuthology.orchestra.run.trial012.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-26T03:21:26.715 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:26.725 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 136/160 2026-02-26T03:21:26.748 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 136/160 2026-02-26T03:21:26.748 INFO:teuthology.orchestra.run.trial012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:21:26.748 INFO:teuthology.orchestra.run.trial012.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-26T03:21:26.748 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-26T03:21:26.748 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-26T03:21:26.748 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:26.806 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: cephadm-2:20.3.0-5312.gabffb617.el9.noarch 137/160 2026-02-26T03:21:26.808 INFO:teuthology.orchestra.run.trial012.stdout: Installing : cephadm-2:20.3.0-5312.gabffb617.el9.noarch 137/160 2026-02-26T03:21:26.817 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.e 138/160 2026-02-26T03:21:26.841 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5312.gabffb617. 139/160 2026-02-26T03:21:26.842 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 140/160 2026-02-26T03:21:29.707 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 140/160 2026-02-26T03:21:29.713 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 141/160 2026-02-26T03:21:30.164 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 141/160 2026-02-26T03:21:30.165 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 142/160 2026-02-26T03:21:30.177 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 142/160 2026-02-26T03:21:30.178 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 143/160 2026-02-26T03:21:30.237 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 143/160 2026-02-26T03:21:30.277 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el 144/160 2026-02-26T03:21:30.278 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 145/160 2026-02-26T03:21:30.299 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 145/160 2026-02-26T03:21:30.299 INFO:teuthology.orchestra.run.trial012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:21:30.300 INFO:teuthology.orchestra.run.trial012.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-26T03:21:30.300 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-26T03:21:30.300 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-26T03:21:30.300 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:30.310 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 146/160 2026-02-26T03:21:32.536 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 146/160 2026-02-26T03:21:55.040 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86 147/160 2026-02-26T03:21:55.041 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb6 148/160 2026-02-26T03:21:55.525 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-2:20.3.0-5312.gabffb617.el9.x86_64 149/160 2026-02-26T03:21:55.527 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 150/160 2026-02-26T03:21:55.549 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 150/160 2026-02-26T03:21:55.549 INFO:teuthology.orchestra.run.trial012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:21:55.549 INFO:teuthology.orchestra.run.trial012.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-26T03:21:55.549 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-26T03:21:55.549 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-26T03:21:55.549 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:55.559 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5312.gabffb 151/160 2026-02-26T03:21:55.578 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5312.gabffb 151/160 2026-02-26T03:21:55.579 INFO:teuthology.orchestra.run.trial012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:21:55.579 INFO:teuthology.orchestra.run.trial012.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-26T03:21:55.579 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:21:55.690 INFO:teuthology.orchestra.run.trial012.stdout: Installing : rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 152/160 2026-02-26T03:21:55.710 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 152/160 2026-02-26T03:21:55.710 INFO:teuthology.orchestra.run.trial012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:21:55.710 INFO:teuthology.orchestra.run.trial012.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-26T03:21:55.710 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-26T03:21:55.710 INFO:teuthology.orchestra.run.trial012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-26T03:21:55.710 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:22:02.676 INFO:teuthology.orchestra.run.trial012.stdout: Installing : python3-pytest-6.2.2-7.el9.noarch 153/160 2026-02-26T03:22:02.684 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 154/160 2026-02-26T03:22:02.689 INFO:teuthology.orchestra.run.trial012.stdout: Installing : libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_ 155/160 2026-02-26T03:22:02.697 INFO:teuthology.orchestra.run.trial012.stdout: Installing : rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 156/160 2026-02-26T03:22:02.738 INFO:teuthology.orchestra.run.trial012.stdout: Installing : rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 157/160 2026-02-26T03:22:02.741 INFO:teuthology.orchestra.run.trial012.stdout: Installing : ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 158/160 2026-02-26T03:22:02.741 INFO:teuthology.orchestra.run.trial012.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/160 2026-02-26T03:22:02.754 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/160 2026-02-26T03:22:02.754 INFO:teuthology.orchestra.run.trial012.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:22:02.764 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:22:02.770 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 160/160 2026-02-26T03:22:02.796 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 160/160 2026-02-26T03:22:06.207 INFO:teuthology.orchestra.run.trial012.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-2:20.3.0-5312.gabffb617.el9.x86_64 1/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 2/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 3/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86 4/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5312.gabffb 6/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 7/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 8/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 9/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 10/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 11/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 12/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb6 13/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 14/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 15/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86 17/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_ 18/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 19/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 20/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 21/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : librados-devel-2:20.3.0-5312.gabffb617.el9.x86_6 22/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 23/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 24/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5312.gabffb617.el 25/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-ceph-common-2:20.3.0-5312.gabffb617.el9. 26/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_6 27/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 28/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 29/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 30/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 31/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 32/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 33/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5312.gabffb617. 34/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 35/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 36/160 2026-02-26T03:22:06.208 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 37/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 38/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el 39/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 40/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.e 41/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 42/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : cephadm-2:20.3.0-5312.gabffb617.el9.noarch 43/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 44/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 45/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 46/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 47/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 48/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 49/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 50/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 51/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 52/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 53/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 54/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 55/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 56/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-ply-3.11-14.el9.noarch 57/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 58/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 59/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 60/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 61/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : unzip-6.0-59.el9.x86_64 62/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : zip-3.0-35.el9.x86_64 63/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 64/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 65/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 66/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 67/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 68/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 69/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 70/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 71/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 72/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 73/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 74/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 75/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 76/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : lua-5.4.4-4.el9.x86_64 77/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 78/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 79/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 80/160 2026-02-26T03:22:06.209 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 81/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 82/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 83/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 84/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 85/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 86/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 87/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 88/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 89/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 90/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 91/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 92/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 93/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 94/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 95/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 96/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-pytest-6.2.2-7.el9.noarch 97/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 98/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 99/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 100/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 101/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 102/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 103/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 104/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 105/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 106/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 107/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 108/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 109/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 110/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 111/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 112/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 113/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 114/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 115/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 116/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 117/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 118/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 119/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 120/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 121/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 122/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 123/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 124/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 125/160 2026-02-26T03:22:06.210 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 126/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 127/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 128/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 129/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 130/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 131/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 132/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 133/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 134/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 135/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 136/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 137/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 138/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 139/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 140/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 141/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 142/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 143/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 144/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 145/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 146/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 147/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 148/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 149/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 150/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 151/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 152/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 153/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : re2-1:20211101-20.el9.x86_64 154/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 155/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 156/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : librados2-2:20.3.0-5312.gabffb617.el9.x86_64 157/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 158/160 2026-02-26T03:22:06.211 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 159/160 2026-02-26T03:22:06.290 INFO:teuthology.orchestra.run.trial012.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:22:06.290 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:22:06.291 INFO:teuthology.orchestra.run.trial012.stdout:Upgraded: 2026-02-26T03:22:06.291 INFO:teuthology.orchestra.run.trial012.stdout: librados2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.291 INFO:teuthology.orchestra.run.trial012.stdout: librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.291 INFO:teuthology.orchestra.run.trial012.stdout:Installed: 2026-02-26T03:22:06.291 INFO:teuthology.orchestra.run.trial012.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-26T03:22:06.291 INFO:teuthology.orchestra.run.trial012.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-26T03:22:06.291 INFO:teuthology.orchestra.run.trial012.stdout: ceph-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.291 INFO:teuthology.orchestra.run.trial012.stdout: ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.291 INFO:teuthology.orchestra.run.trial012.stdout: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.291 INFO:teuthology.orchestra.run.trial012.stdout: ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-grafana-dashboards-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-immutable-object-cache-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:22:06.292 INFO:teuthology.orchestra.run.trial012.stdout: ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: cephadm-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-26T03:22:06.293 INFO:teuthology.orchestra.run.trial012.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.294 INFO:teuthology.orchestra.run.trial012.stdout: libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: librados-devel-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.295 INFO:teuthology.orchestra.run.trial012.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-26T03:22:06.296 INFO:teuthology.orchestra.run.trial012.stdout: librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.296 INFO:teuthology.orchestra.run.trial012.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-26T03:22:06.296 INFO:teuthology.orchestra.run.trial012.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-26T03:22:06.296 INFO:teuthology.orchestra.run.trial012.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-26T03:22:06.296 INFO:teuthology.orchestra.run.trial012.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-26T03:22:06.296 INFO:teuthology.orchestra.run.trial012.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-26T03:22:06.296 INFO:teuthology.orchestra.run.trial012.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-26T03:22:06.296 INFO:teuthology.orchestra.run.trial012.stdout: lua-5.4.4-4.el9.x86_64 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-26T03:22:06.297 INFO:teuthology.orchestra.run.trial012.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-ceph-argparse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-26T03:22:06.298 INFO:teuthology.orchestra.run.trial012.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-26T03:22:06.299 INFO:teuthology.orchestra.run.trial012.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-26T03:22:06.299 INFO:teuthology.orchestra.run.trial012.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-26T03:22:06.299 INFO:teuthology.orchestra.run.trial012.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-26T03:22:06.299 INFO:teuthology.orchestra.run.trial012.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-26T03:22:06.299 INFO:teuthology.orchestra.run.trial012.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2026-02-26T03:22:06.299 INFO:teuthology.orchestra.run.trial012.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-26T03:22:06.299 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-26T03:22:06.299 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-26T03:22:06.299 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-26T03:22:06.299 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-26T03:22:06.299 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-26T03:22:06.300 INFO:teuthology.orchestra.run.trial012.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-pluggy-0.13.1-7.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-ply-3.11-14.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-py-1.10.0-6.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-pytest-6.2.2-7.el9.noarch 2026-02-26T03:22:06.301 INFO:teuthology.orchestra.run.trial012.stdout: python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-26T03:22:06.302 INFO:teuthology.orchestra.run.trial012.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: re2-1:20211101-20.el9.x86_64 2026-02-26T03:22:06.303 INFO:teuthology.orchestra.run.trial012.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-26T03:22:06.304 INFO:teuthology.orchestra.run.trial012.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-26T03:22:06.304 INFO:teuthology.orchestra.run.trial012.stdout: unzip-6.0-59.el9.x86_64 2026-02-26T03:22:06.304 INFO:teuthology.orchestra.run.trial012.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-26T03:22:06.304 INFO:teuthology.orchestra.run.trial012.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-26T03:22:06.304 INFO:teuthology.orchestra.run.trial012.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-26T03:22:06.304 INFO:teuthology.orchestra.run.trial012.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-26T03:22:06.304 INFO:teuthology.orchestra.run.trial012.stdout: zip-3.0-35.el9.x86_64 2026-02-26T03:22:06.304 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:22:06.304 INFO:teuthology.orchestra.run.trial012.stdout:Complete! 2026-02-26T03:22:06.448 DEBUG:teuthology.parallel:result is None 2026-02-26T03:24:33.645 INFO:teuthology.orchestra.run.trial068.stdout:(158/158): ceph-osd-crimson-debuginfo-20.3.0-53 618 kB/s | 429 MB 11:51 2026-02-26T03:24:33.647 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-26T03:24:33.647 INFO:teuthology.orchestra.run.trial068.stdout:Total 1.2 MB/s | 863 MB 12:13 2026-02-26T03:24:34.190 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-26T03:24:34.243 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-26T03:24:34.243 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-26T03:24:36.999 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-26T03:24:36.999 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-26T03:24:39.814 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-26T03:24:39.817 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/160 2026-02-26T03:24:39.826 INFO:teuthology.orchestra.run.trial068.stdout: Installing : liboath-2.6.12-1.el9.x86_64 2/160 2026-02-26T03:24:39.837 INFO:teuthology.orchestra.run.trial068.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/160 2026-02-26T03:24:39.978 INFO:teuthology.orchestra.run.trial068.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2026-02-26T03:24:39.980 INFO:teuthology.orchestra.run.trial068.stdout: Upgrading : librados2-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:24:40.025 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librados2-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:24:40.026 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 6/160 2026-02-26T03:24:40.037 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 6/160 2026-02-26T03:24:40.039 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86 7/160 2026-02-26T03:24:40.040 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 8/160 2026-02-26T03:24:40.068 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 8/160 2026-02-26T03:24:40.073 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 9/160 2026-02-26T03:24:40.082 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 10/160 2026-02-26T03:24:40.085 INFO:teuthology.orchestra.run.trial068.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 11/160 2026-02-26T03:24:40.088 INFO:teuthology.orchestra.run.trial068.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 12/160 2026-02-26T03:24:40.090 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 13/160 2026-02-26T03:24:40.094 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 14/160 2026-02-26T03:24:40.209 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 15/160 2026-02-26T03:24:40.210 INFO:teuthology.orchestra.run.trial068.stdout: Upgrading : librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:24:40.251 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:24:40.257 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 17/160 2026-02-26T03:24:40.267 INFO:teuthology.orchestra.run.trial068.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 18/160 2026-02-26T03:24:40.268 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 19/160 2026-02-26T03:24:40.290 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 19/160 2026-02-26T03:24:40.291 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 20/160 2026-02-26T03:24:40.303 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 20/160 2026-02-26T03:24:40.307 INFO:teuthology.orchestra.run.trial068.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 21/160 2026-02-26T03:24:40.336 INFO:teuthology.orchestra.run.trial068.stdout: Installing : re2-1:20211101-20.el9.x86_64 22/160 2026-02-26T03:24:40.342 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 23/160 2026-02-26T03:24:40.345 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 24/160 2026-02-26T03:24:40.367 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 25/160 2026-02-26T03:24:40.372 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 26/160 2026-02-26T03:24:40.375 INFO:teuthology.orchestra.run.trial068.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2026-02-26T03:24:40.379 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 28/160 2026-02-26T03:24:40.380 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 29/160 2026-02-26T03:24:40.392 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 29/160 2026-02-26T03:24:40.397 INFO:teuthology.orchestra.run.trial068.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 30/160 2026-02-26T03:24:40.422 INFO:teuthology.orchestra.run.trial068.stdout: Installing : unzip-6.0-59.el9.x86_64 31/160 2026-02-26T03:24:40.436 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 32/160 2026-02-26T03:24:40.441 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 33/160 2026-02-26T03:24:40.446 INFO:teuthology.orchestra.run.trial068.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 34/160 2026-02-26T03:24:40.452 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 35/160 2026-02-26T03:24:40.454 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 36/160 2026-02-26T03:24:40.456 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 37/160 2026-02-26T03:24:40.465 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 38/160 2026-02-26T03:24:40.500 INFO:teuthology.orchestra.run.trial068.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 39/160 2026-02-26T03:24:40.504 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-ceph-common-2:20.3.0-5312.gabffb617.el9. 40/160 2026-02-26T03:24:40.515 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-ceph-argparse-2:20.3.0-5312.gabffb617.el 41/160 2026-02-26T03:24:40.526 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_6 42/160 2026-02-26T03:24:40.532 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 43/160 2026-02-26T03:24:40.556 INFO:teuthology.orchestra.run.trial068.stdout: Installing : zip-3.0-35.el9.x86_64 44/160 2026-02-26T03:24:40.559 INFO:teuthology.orchestra.run.trial068.stdout: Installing : luarocks-3.9.2-5.el9.noarch 45/160 2026-02-26T03:24:40.610 INFO:teuthology.orchestra.run.trial068.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 46/160 2026-02-26T03:24:40.623 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 47/160 2026-02-26T03:24:40.630 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-rsa-4.9-2.el9.noarch 48/160 2026-02-26T03:24:40.633 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/160 2026-02-26T03:24:40.636 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/160 2026-02-26T03:24:40.641 INFO:teuthology.orchestra.run.trial068.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 51/160 2026-02-26T03:24:40.657 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 52/160 2026-02-26T03:24:40.662 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 53/160 2026-02-26T03:24:40.666 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/160 2026-02-26T03:24:40.673 INFO:teuthology.orchestra.run.trial068.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 55/160 2026-02-26T03:24:40.680 INFO:teuthology.orchestra.run.trial068.stdout: Installing : librados-devel-2:20.3.0-5312.gabffb617.el9.x86_6 56/160 2026-02-26T03:24:40.692 INFO:teuthology.orchestra.run.trial068.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 57/160 2026-02-26T03:24:40.697 INFO:teuthology.orchestra.run.trial068.stdout: Installing : thrift-0.15.0-4.el9.x86_64 58/160 2026-02-26T03:24:40.700 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 59/160 2026-02-26T03:24:40.715 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 60/160 2026-02-26T03:24:40.719 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 61/160 2026-02-26T03:24:40.725 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 62/160 2026-02-26T03:24:40.737 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 63/160 2026-02-26T03:24:40.746 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 64/160 2026-02-26T03:24:40.753 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 65/160 2026-02-26T03:24:40.772 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 66/160 2026-02-26T03:24:40.783 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 67/160 2026-02-26T03:24:40.799 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 68/160 2026-02-26T03:24:40.804 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 69/160 2026-02-26T03:24:40.812 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 70/160 2026-02-26T03:24:40.852 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 71/160 2026-02-26T03:24:41.179 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 72/160 2026-02-26T03:24:41.189 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 73/160 2026-02-26T03:24:41.193 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 74/160 2026-02-26T03:24:41.199 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 75/160 2026-02-26T03:24:41.205 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 76/160 2026-02-26T03:24:41.208 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 77/160 2026-02-26T03:24:41.211 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/160 2026-02-26T03:24:41.217 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 79/160 2026-02-26T03:24:41.220 INFO:teuthology.orchestra.run.trial068.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 80/160 2026-02-26T03:24:41.221 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 81/160 2026-02-26T03:24:41.250 INFO:teuthology.orchestra.run.trial068.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 82/160 2026-02-26T03:24:41.294 INFO:teuthology.orchestra.run.trial068.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 83/160 2026-02-26T03:24:41.303 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 84/160 2026-02-26T03:24:41.350 INFO:teuthology.orchestra.run.trial068.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 85/160 2026-02-26T03:24:41.379 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-py-1.10.0-6.el9.noarch 86/160 2026-02-26T03:24:41.389 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 87/160 2026-02-26T03:24:41.396 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 88/160 2026-02-26T03:24:41.401 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 89/160 2026-02-26T03:24:41.408 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 90/160 2026-02-26T03:24:41.412 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 91/160 2026-02-26T03:24:41.447 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 92/160 2026-02-26T03:24:41.673 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 93/160 2026-02-26T03:24:41.697 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 94/160 2026-02-26T03:24:41.700 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 95/160 2026-02-26T03:24:41.759 INFO:teuthology.orchestra.run.trial068.stdout: Installing : openblas-0.3.29-1.el9.x86_64 96/160 2026-02-26T03:24:41.761 INFO:teuthology.orchestra.run.trial068.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 97/160 2026-02-26T03:24:41.782 INFO:teuthology.orchestra.run.trial068.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 98/160 2026-02-26T03:24:42.111 INFO:teuthology.orchestra.run.trial068.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 99/160 2026-02-26T03:24:42.184 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 100/160 2026-02-26T03:24:42.861 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 101/160 2026-02-26T03:24:42.875 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 102/160 2026-02-26T03:24:42.881 INFO:teuthology.orchestra.run.trial068.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 103/160 2026-02-26T03:24:43.128 INFO:teuthology.orchestra.run.trial068.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 104/160 2026-02-26T03:24:43.129 INFO:teuthology.orchestra.run.trial068.stdout: Installing : librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 105/160 2026-02-26T03:24:43.147 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 105/160 2026-02-26T03:24:43.148 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 106/160 2026-02-26T03:24:44.088 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:24:44.090 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:24:44.106 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:24:44.116 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 108/160 2026-02-26T03:24:44.123 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-packaging-20.9-5.el9.noarch 109/160 2026-02-26T03:24:44.136 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-ply-3.11-14.el9.noarch 110/160 2026-02-26T03:24:44.152 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 111/160 2026-02-26T03:24:44.228 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 112/160 2026-02-26T03:24:44.237 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 113/160 2026-02-26T03:24:44.261 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 114/160 2026-02-26T03:24:44.290 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 115/160 2026-02-26T03:24:44.341 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 116/160 2026-02-26T03:24:44.347 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 117/160 2026-02-26T03:24:44.351 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 118/160 2026-02-26T03:24:44.356 INFO:teuthology.orchestra.run.trial068.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 119/160 2026-02-26T03:24:44.358 INFO:teuthology.orchestra.run.trial068.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 120/160 2026-02-26T03:24:44.359 INFO:teuthology.orchestra.run.trial068.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 121/160 2026-02-26T03:24:44.375 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 121/160 2026-02-26T03:24:44.489 INFO:teuthology.orchestra.run.trial068.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 122/160 2026-02-26T03:24:44.491 INFO:teuthology.orchestra.run.trial068.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 123/160 2026-02-26T03:24:44.786 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 123/160 2026-02-26T03:24:44.787 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-26T03:24:44.787 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:24:45.042 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 124/160 2026-02-26T03:24:45.083 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 124/160 2026-02-26T03:24:45.083 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-26T03:24:45.084 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-26T03:24:45.084 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:24:45.087 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 125/160 2026-02-26T03:24:49.746 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 125/160 2026-02-26T03:24:49.747 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /sys 2026-02-26T03:24:49.747 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /proc 2026-02-26T03:24:49.747 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /mnt 2026-02-26T03:24:49.747 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /var/tmp 2026-02-26T03:24:49.747 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /home 2026-02-26T03:24:49.747 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /root 2026-02-26T03:24:49.747 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /tmp 2026-02-26T03:24:49.747 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:24:49.833 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 126/160 2026-02-26T03:24:49.855 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 126/160 2026-02-26T03:24:49.855 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:24:49.855 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-26T03:24:49.855 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-26T03:24:49.855 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-26T03:24:49.855 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:24:50.058 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 127/160 2026-02-26T03:24:50.078 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 127/160 2026-02-26T03:24:50.078 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:24:50.079 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-26T03:24:50.079 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-26T03:24:50.079 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-26T03:24:50.079 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:24:50.084 INFO:teuthology.orchestra.run.trial068.stdout: Installing : mailcap-2.1.49-5.el9.noarch 128/160 2026-02-26T03:24:50.086 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 129/160 2026-02-26T03:24:50.103 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:24:50.103 INFO:teuthology.orchestra.run.trial068.stdout:Creating group 'qat' with GID 992. 2026-02-26T03:24:50.103 INFO:teuthology.orchestra.run.trial068.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-26T03:24:50.103 INFO:teuthology.orchestra.run.trial068.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-26T03:24:50.103 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:24:50.110 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:24:50.132 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:24:50.132 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-26T03:24:50.132 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:24:50.166 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 131/160 2026-02-26T03:24:50.749 INFO:teuthology.orchestra.run.trial068.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 132/160 2026-02-26T03:24:56.082 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 133/160 2026-02-26T03:24:56.145 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 134/160 2026-02-26T03:24:56.147 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 135/160 2026-02-26T03:24:56.160 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 135/160 2026-02-26T03:24:56.160 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:24:56.161 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-26T03:24:56.161 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:24:56.172 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 136/160 2026-02-26T03:24:56.195 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 136/160 2026-02-26T03:24:56.195 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:24:56.195 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-26T03:24:56.195 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-26T03:24:56.195 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-26T03:24:56.195 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:24:56.252 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: cephadm-2:20.3.0-5312.gabffb617.el9.noarch 137/160 2026-02-26T03:24:56.254 INFO:teuthology.orchestra.run.trial068.stdout: Installing : cephadm-2:20.3.0-5312.gabffb617.el9.noarch 137/160 2026-02-26T03:24:56.263 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.e 138/160 2026-02-26T03:24:56.287 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5312.gabffb617. 139/160 2026-02-26T03:24:56.288 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 140/160 2026-02-26T03:24:59.157 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 140/160 2026-02-26T03:24:59.163 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 141/160 2026-02-26T03:24:59.617 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 141/160 2026-02-26T03:24:59.618 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 142/160 2026-02-26T03:24:59.631 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 142/160 2026-02-26T03:24:59.631 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 143/160 2026-02-26T03:24:59.692 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 143/160 2026-02-26T03:24:59.732 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el 144/160 2026-02-26T03:24:59.733 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 145/160 2026-02-26T03:24:59.753 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 145/160 2026-02-26T03:24:59.753 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:24:59.753 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-26T03:24:59.753 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-26T03:24:59.753 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-26T03:24:59.753 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:24:59.764 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 146/160 2026-02-26T03:25:02.013 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 146/160 2026-02-26T03:25:24.068 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86 147/160 2026-02-26T03:25:24.069 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb6 148/160 2026-02-26T03:25:24.550 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-2:20.3.0-5312.gabffb617.el9.x86_64 149/160 2026-02-26T03:25:24.552 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 150/160 2026-02-26T03:25:24.573 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 150/160 2026-02-26T03:25:24.573 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:25:24.573 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-26T03:25:24.574 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-26T03:25:24.574 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-26T03:25:24.574 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:24.582 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5312.gabffb 151/160 2026-02-26T03:25:24.601 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5312.gabffb 151/160 2026-02-26T03:25:24.601 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:25:24.601 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-26T03:25:24.601 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:24.709 INFO:teuthology.orchestra.run.trial068.stdout: Installing : rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 152/160 2026-02-26T03:25:24.728 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 152/160 2026-02-26T03:25:24.728 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:25:24.728 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-26T03:25:24.728 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-26T03:25:24.728 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-26T03:25:24.728 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:31.632 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pytest-6.2.2-7.el9.noarch 153/160 2026-02-26T03:25:31.641 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 154/160 2026-02-26T03:25:31.645 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_ 155/160 2026-02-26T03:25:31.654 INFO:teuthology.orchestra.run.trial068.stdout: Installing : rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 156/160 2026-02-26T03:25:31.695 INFO:teuthology.orchestra.run.trial068.stdout: Installing : rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 157/160 2026-02-26T03:25:31.698 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 158/160 2026-02-26T03:25:31.698 INFO:teuthology.orchestra.run.trial068.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/160 2026-02-26T03:25:31.710 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/160 2026-02-26T03:25:31.711 INFO:teuthology.orchestra.run.trial068.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:25:31.721 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:25:31.727 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 160/160 2026-02-26T03:25:31.751 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 160/160 2026-02-26T03:25:34.877 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:25:34.877 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-2:20.3.0-5312.gabffb617.el9.x86_64 1/160 2026-02-26T03:25:34.877 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 2/160 2026-02-26T03:25:34.877 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 3/160 2026-02-26T03:25:34.877 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86 4/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5312.gabffb 6/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 7/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 8/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 9/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 10/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 11/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 12/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb6 13/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 14/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 15/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:25:34.878 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86 17/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_ 18/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 19/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 20/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 21/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librados-devel-2:20.3.0-5312.gabffb617.el9.x86_6 22/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 23/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 24/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5312.gabffb617.el 25/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-ceph-common-2:20.3.0-5312.gabffb617.el9. 26/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_6 27/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 28/160 2026-02-26T03:25:34.879 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 29/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 30/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 31/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 32/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 33/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5312.gabffb617. 34/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 35/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 36/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 37/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 38/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el 39/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 40/160 2026-02-26T03:25:34.880 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.e 41/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 42/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : cephadm-2:20.3.0-5312.gabffb617.el9.noarch 43/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 44/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 45/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 46/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 47/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 48/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 49/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 50/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 51/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 52/160 2026-02-26T03:25:34.881 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 53/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 54/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 55/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 56/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-ply-3.11-14.el9.noarch 57/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 58/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 59/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 60/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 61/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : unzip-6.0-59.el9.x86_64 62/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : zip-3.0-35.el9.x86_64 63/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 64/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 65/160 2026-02-26T03:25:34.882 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 66/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 67/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 68/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 69/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 70/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 71/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 72/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 73/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 74/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 75/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 76/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : lua-5.4.4-4.el9.x86_64 77/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 78/160 2026-02-26T03:25:34.883 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 79/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 80/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 81/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 82/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 83/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 84/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 85/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 86/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 87/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 88/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 89/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 90/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 91/160 2026-02-26T03:25:34.884 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 92/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 93/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 94/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 95/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 96/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pytest-6.2.2-7.el9.noarch 97/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 98/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 99/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 100/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 101/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 102/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 103/160 2026-02-26T03:25:34.885 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 104/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 105/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 106/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 107/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 108/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 109/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 110/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 111/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 112/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 113/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 114/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 115/160 2026-02-26T03:25:34.886 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 116/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 117/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 118/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 119/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 120/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 121/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 122/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 123/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 124/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 125/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 126/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 127/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 128/160 2026-02-26T03:25:34.887 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 129/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 130/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 131/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 132/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 133/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 134/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 135/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 136/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 137/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 138/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 139/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 140/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 141/160 2026-02-26T03:25:34.888 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 142/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 143/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 144/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 145/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 146/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 147/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 148/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 149/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 150/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 151/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 152/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 153/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : re2-1:20211101-20.el9.x86_64 154/160 2026-02-26T03:25:34.889 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 155/160 2026-02-26T03:25:34.890 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 156/160 2026-02-26T03:25:34.890 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librados2-2:20.3.0-5312.gabffb617.el9.x86_64 157/160 2026-02-26T03:25:34.890 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 158/160 2026-02-26T03:25:34.890 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 159/160 2026-02-26T03:25:34.949 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:25:34.949 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:34.949 INFO:teuthology.orchestra.run.trial068.stdout:Upgraded: 2026-02-26T03:25:34.949 INFO:teuthology.orchestra.run.trial068.stdout: librados2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.949 INFO:teuthology.orchestra.run.trial068.stdout: librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.949 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-26T03:25:34.949 INFO:teuthology.orchestra.run.trial068.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: ceph-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: ceph-grafana-dashboards-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: ceph-immutable-object-cache-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:25:34.950 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:25:34.951 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:25:34.951 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:25:34.951 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:25:34.951 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:25:34.951 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.951 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.951 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.951 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.951 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.951 INFO:teuthology.orchestra.run.trial068.stdout: ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:25:34.951 INFO:teuthology.orchestra.run.trial068.stdout: ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: cephadm-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-26T03:25:34.952 INFO:teuthology.orchestra.run.trial068.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-26T03:25:34.953 INFO:teuthology.orchestra.run.trial068.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: librados-devel-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-26T03:25:34.954 INFO:teuthology.orchestra.run.trial068.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: lua-5.4.4-4.el9.x86_64 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-26T03:25:34.955 INFO:teuthology.orchestra.run.trial068.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-ceph-argparse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-26T03:25:34.956 INFO:teuthology.orchestra.run.trial068.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-26T03:25:34.957 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-26T03:25:34.958 INFO:teuthology.orchestra.run.trial068.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-pluggy-0.13.1-7.el9.noarch 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-ply-3.11-14.el9.noarch 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-py-1.10.0-6.el9.noarch 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-26T03:25:34.959 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-pytest-6.2.2-7.el9.noarch 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-26T03:25:34.960 INFO:teuthology.orchestra.run.trial068.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.961 INFO:teuthology.orchestra.run.trial068.stdout: rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout: rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout: re2-1:20211101-20.el9.x86_64 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout: unzip-6.0-59.el9.x86_64 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout: zip-3.0-35.el9.x86_64 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:34.962 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-26T03:25:35.100 DEBUG:teuthology.parallel:result is None 2026-02-26T03:25:35.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:25:35.238 DEBUG:teuthology.orchestra.run.trial012:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-26T03:25:35.271 INFO:teuthology.orchestra.run.trial012.stdout:20.3.0-5312.gabffb617.el9 2026-02-26T03:25:35.272 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5312.gabffb617.el9 2026-02-26T03:25:35.272 INFO:teuthology.task.install:The correct ceph version 20.3.0-5312.gabffb617 is installed. 2026-02-26T03:25:35.273 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:25:35.405 DEBUG:teuthology.orchestra.run.trial068:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-26T03:25:35.428 INFO:teuthology.orchestra.run.trial068.stdout:20.3.0-5312.gabffb617.el9 2026-02-26T03:25:35.428 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5312.gabffb617.el9 2026-02-26T03:25:35.428 INFO:teuthology.task.install:The correct ceph version 20.3.0-5312.gabffb617 is installed. 2026-02-26T03:25:35.430 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-26T03:25:35.431 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:25:35.431 DEBUG:teuthology.orchestra.run.trial012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-26T03:25:35.459 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:25:35.459 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-26T03:25:35.494 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-26T03:25:35.494 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:25:35.494 DEBUG:teuthology.orchestra.run.trial012:> sudo dd of=/usr/bin/daemon-helper 2026-02-26T03:25:35.522 DEBUG:teuthology.orchestra.run.trial012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-26T03:25:35.585 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:25:35.585 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/usr/bin/daemon-helper 2026-02-26T03:25:35.610 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-26T03:25:35.672 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-26T03:25:35.672 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:25:35.672 DEBUG:teuthology.orchestra.run.trial012:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-26T03:25:35.697 DEBUG:teuthology.orchestra.run.trial012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-26T03:25:35.761 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:25:35.761 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-26T03:25:35.785 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-26T03:25:35.849 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-26T03:25:35.849 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:25:35.850 DEBUG:teuthology.orchestra.run.trial012:> sudo dd of=/usr/bin/stdin-killer 2026-02-26T03:25:35.875 DEBUG:teuthology.orchestra.run.trial012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-26T03:25:35.938 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:25:35.938 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/usr/bin/stdin-killer 2026-02-26T03:25:35.963 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-26T03:25:36.027 INFO:teuthology.run_tasks:Running task pexec... 2026-02-26T03:25:36.046 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-26T03:25:36.046 DEBUG:teuthology.orchestra.run.trial012:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-26T03:25:36.047 DEBUG:teuthology.orchestra.run.trial068:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-26T03:25:36.050 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial012.front.sepia.ceph.com 2026-02-26T03:25:36.050 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-26T03:25:36.070 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial068.front.sepia.ceph.com 2026-02-26T03:25:36.070 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-26T03:25:36.100 DEBUG:teuthology.parallel:result is None 2026-02-26T03:25:36.121 DEBUG:teuthology.parallel:result is None 2026-02-26T03:25:36.121 INFO:teuthology.run_tasks:Running task install... 2026-02-26T03:25:36.137 DEBUG:teuthology.task.install:project ceph 2026-02-26T03:25:36.137 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'}} 2026-02-26T03:25:36.137 DEBUG:teuthology.task.install:config {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'} 2026-02-26T03:25:36.137 INFO:teuthology.task.install:Using flavor: debug 2026-02-26T03:25:36.146 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-26T03:25:36.147 INFO:teuthology.task.install:extra packages: [] 2026-02-26T03:25:36.147 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'tag': None, 'wait_for_package': False} 2026-02-26T03:25:36.147 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:25:36.149 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'tag': None, 'wait_for_package': False} 2026-02-26T03:25:36.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:25:36.340 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/ 2026-02-26T03:25:36.340 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5312.gabffb617 2026-02-26T03:25:36.351 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/ 2026-02-26T03:25:36.351 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5312.gabffb617 2026-02-26T03:25:36.716 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-26T03:25:36.717 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:25:36.717 DEBUG:teuthology.orchestra.run.trial012:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-26T03:25:36.742 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest on remote rpm x86_64 2026-02-26T03:25:36.742 DEBUG:teuthology.orchestra.run.trial012:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-26T03:25:36.768 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-26T03:25:36.768 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:25:36.768 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-26T03:25:36.793 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest on remote rpm x86_64 2026-02-26T03:25:36.793 DEBUG:teuthology.orchestra.run.trial068:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-26T03:25:36.806 DEBUG:teuthology.orchestra.run.trial012:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-26T03:25:36.858 DEBUG:teuthology.orchestra.run.trial068:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-26T03:25:36.869 DEBUG:teuthology.orchestra.run.trial012:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-26T03:25:36.922 DEBUG:teuthology.orchestra.run.trial068:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-26T03:25:36.924 INFO:teuthology.orchestra.run.trial012.stdout:check_obsoletes = 1 2026-02-26T03:25:36.934 DEBUG:teuthology.orchestra.run.trial012:> sudo yum clean all 2026-02-26T03:25:36.979 INFO:teuthology.orchestra.run.trial068.stdout:check_obsoletes = 1 2026-02-26T03:25:36.989 DEBUG:teuthology.orchestra.run.trial068:> sudo yum clean all 2026-02-26T03:25:37.092 INFO:teuthology.orchestra.run.trial012.stdout:56 files removed 2026-02-26T03:25:37.108 DEBUG:teuthology.orchestra.run.trial012:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest 2026-02-26T03:25:37.150 INFO:teuthology.orchestra.run.trial068.stdout:56 files removed 2026-02-26T03:25:37.166 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest 2026-02-26T03:25:37.989 INFO:teuthology.orchestra.run.trial012.stdout:ceph packages for x86_64 130 kB/s | 94 kB 00:00 2026-02-26T03:25:38.053 INFO:teuthology.orchestra.run.trial068.stdout:ceph packages for x86_64 128 kB/s | 94 kB 00:00 2026-02-26T03:25:38.669 INFO:teuthology.orchestra.run.trial012.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2026-02-26T03:25:38.733 INFO:teuthology.orchestra.run.trial068.stdout:ceph noarch packages 38 kB/s | 24 kB 00:00 2026-02-26T03:25:39.263 INFO:teuthology.orchestra.run.trial012.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-26T03:25:39.330 INFO:teuthology.orchestra.run.trial068.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-26T03:25:40.917 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.9 MB 00:01 2026-02-26T03:25:41.040 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.9 MB 00:01 2026-02-26T03:25:44.320 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 27 MB 00:02 2026-02-26T03:25:45.381 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - AppStream 7.1 MB/s | 27 MB 00:03 2026-02-26T03:25:48.258 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 8.0 MB 00:01 2026-02-26T03:25:48.729 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 8.0 MB 00:00 2026-02-26T03:25:49.141 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - Extras packages 139 kB/s | 20 kB 00:00 2026-02-26T03:25:50.174 INFO:teuthology.orchestra.run.trial068.stdout:Extra Packages for Enterprise Linux 21 MB/s | 20 MB 00:00 2026-02-26T03:25:51.451 INFO:teuthology.orchestra.run.trial012.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 20 kB 00:01 2026-02-26T03:25:52.743 INFO:teuthology.orchestra.run.trial012.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2026-02-26T03:25:53.409 INFO:teuthology.orchestra.run.trial068.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-26T03:25:54.618 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.619 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.619 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.620 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.620 INFO:teuthology.orchestra.run.trial068.stdout:Package cephadm-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:54.620 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-immutable-object-cache-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.620 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.621 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:54.621 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:54.622 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:54.622 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:54.622 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.623 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.623 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.624 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:54.624 INFO:teuthology.orchestra.run.trial068.stdout:Package librados-devel-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.625 INFO:teuthology.orchestra.run.trial068.stdout:Package libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.625 INFO:teuthology.orchestra.run.trial068.stdout:Package libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.626 INFO:teuthology.orchestra.run.trial068.stdout:Package librados2-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.626 INFO:teuthology.orchestra.run.trial068.stdout:Package librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.626 INFO:teuthology.orchestra.run.trial068.stdout:Package python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.627 INFO:teuthology.orchestra.run.trial068.stdout:Package python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.627 INFO:teuthology.orchestra.run.trial068.stdout:Package python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.628 INFO:teuthology.orchestra.run.trial068.stdout:Package python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.628 INFO:teuthology.orchestra.run.trial068.stdout:Package rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.629 INFO:teuthology.orchestra.run.trial068.stdout:Package rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.629 INFO:teuthology.orchestra.run.trial068.stdout:Package rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:54.629 INFO:teuthology.orchestra.run.trial068.stdout:Package python3-pytest-6.2.2-7.el9.noarch is already installed. 2026-02-26T03:25:54.648 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-26T03:25:54.648 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-26T03:25:54.648 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-26T03:25:54.676 DEBUG:teuthology.parallel:result is None 2026-02-26T03:25:55.963 INFO:teuthology.orchestra.run.trial012.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-26T03:25:57.161 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.162 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.162 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.162 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.162 INFO:teuthology.orchestra.run.trial012.stdout:Package cephadm-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:57.163 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-immutable-object-cache-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.163 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.164 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:57.164 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:57.165 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:57.165 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:57.165 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.166 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.166 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.167 INFO:teuthology.orchestra.run.trial012.stdout:Package ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:25:57.167 INFO:teuthology.orchestra.run.trial012.stdout:Package librados-devel-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.168 INFO:teuthology.orchestra.run.trial012.stdout:Package libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.168 INFO:teuthology.orchestra.run.trial012.stdout:Package libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.168 INFO:teuthology.orchestra.run.trial012.stdout:Package librados2-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.169 INFO:teuthology.orchestra.run.trial012.stdout:Package librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.169 INFO:teuthology.orchestra.run.trial012.stdout:Package python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.170 INFO:teuthology.orchestra.run.trial012.stdout:Package python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.170 INFO:teuthology.orchestra.run.trial012.stdout:Package python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.171 INFO:teuthology.orchestra.run.trial012.stdout:Package python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.171 INFO:teuthology.orchestra.run.trial012.stdout:Package rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.171 INFO:teuthology.orchestra.run.trial012.stdout:Package rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.172 INFO:teuthology.orchestra.run.trial012.stdout:Package rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:25:57.172 INFO:teuthology.orchestra.run.trial012.stdout:Package python3-pytest-6.2.2-7.el9.noarch is already installed. 2026-02-26T03:25:57.191 INFO:teuthology.orchestra.run.trial012.stdout:Dependencies resolved. 2026-02-26T03:25:57.191 INFO:teuthology.orchestra.run.trial012.stdout:Nothing to do. 2026-02-26T03:25:57.192 INFO:teuthology.orchestra.run.trial012.stdout:Complete! 2026-02-26T03:25:57.219 DEBUG:teuthology.parallel:result is None 2026-02-26T03:25:57.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:25:57.392 DEBUG:teuthology.orchestra.run.trial012:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-26T03:25:57.413 INFO:teuthology.orchestra.run.trial012.stdout:20.3.0-5312.gabffb617.el9 2026-02-26T03:25:57.413 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5312.gabffb617.el9 2026-02-26T03:25:57.413 INFO:teuthology.task.install:The correct ceph version 20.3.0-5312.gabffb617 is installed. 2026-02-26T03:25:57.416 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:25:57.566 DEBUG:teuthology.orchestra.run.trial068:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-26T03:25:57.587 INFO:teuthology.orchestra.run.trial068.stdout:20.3.0-5312.gabffb617.el9 2026-02-26T03:25:57.588 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5312.gabffb617.el9 2026-02-26T03:25:57.588 INFO:teuthology.task.install:The correct ceph version 20.3.0-5312.gabffb617 is installed. 2026-02-26T03:25:57.591 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-26T03:25:57.591 INFO:teuthology.run_tasks:Running task ceph... 2026-02-26T03:25:57.688 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-26T03:25:57.688 DEBUG:teuthology.orchestra.run.trial012:> sudo chmod 777 /var/log/ceph 2026-02-26T03:25:57.690 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod 777 /var/log/ceph 2026-02-26T03:25:57.715 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-26T03:25:57.715 DEBUG:teuthology.orchestra.run.trial012:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-26T03:25:57.755 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-26T03:25:57.778 INFO:tasks.ceph:Creating extra log directories... 2026-02-26T03:25:57.778 DEBUG:teuthology.orchestra.run.trial012:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-26T03:25:57.820 DEBUG:teuthology.orchestra.run.trial068:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-26T03:25:57.859 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-26T03:25:57.859 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-26T03:25:57.859 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71166', 'branch': 'main', 'description': 'crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rados_python}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '71166', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996'}}, 'owner': 'scheduled_shraddhaag@soko04.front.sepia.ceph.com', 'priority': 101, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5117, 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996', 'targets': {'trial012.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIBq26EaQGQhBUimtfMSyEGHSYLQfmIxPjLSFKjYJdpPFfx1cKeWkRqLO68Vmz2AqfUrmsKMTuInDcDcl4xcH1k=', 'trial068.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN1b3gHvHZqKqXUNBfS4WCaAsVMlt++lKdFheIAW97jHl8QvLS3hEbXawzsN1S9hBsvyVqW8v5EMhNjyr0rjSw4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ["rados/test_python.sh -m 'not (wait or tier or ec)'"]}, 'timeout': '1h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '6e3d0419ccca0e75a03e08feb708946e198236f4', 'timestamp': '2026-02-26_02:22:56', 'tube': 'trial', 'user': 'shraddhaag', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699'} 2026-02-26T03:25:57.860 DEBUG:teuthology.orchestra.run.trial012:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-26T03:25:57.885 DEBUG:teuthology.orchestra.run.trial068:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-26T03:25:57.902 DEBUG:teuthology.orchestra.run.trial012:> sudo install -d -m0777 -- /var/run/ceph 2026-02-26T03:25:57.942 DEBUG:teuthology.orchestra.run.trial068:> sudo install -d -m0777 -- /var/run/ceph 2026-02-26T03:25:57.967 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:25:57.967 DEBUG:teuthology.orchestra.run.trial012:> dd if=/scratch_devs of=/dev/stdout 2026-02-26T03:25:58.021 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-26T03:25:58.021 DEBUG:teuthology.orchestra.run.trial012:> stat /dev/vg_nvme/lv_1 2026-02-26T03:25:58.077 INFO:teuthology.orchestra.run.trial012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-26T03:25:58.077 INFO:teuthology.orchestra.run.trial012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:25:58.077 INFO:teuthology.orchestra.run.trial012.stdout:Device: 6h/6d Inode: 1299 Links: 1 2026-02-26T03:25:58.077 INFO:teuthology.orchestra.run.trial012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:25:58.077 INFO:teuthology.orchestra.run.trial012.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:25:58.077 INFO:teuthology.orchestra.run.trial012.stdout:Access: 2026-02-26 03:22:02.991678904 +0000 2026-02-26T03:25:58.077 INFO:teuthology.orchestra.run.trial012.stdout:Modify: 2026-02-26 03:21:15.321054664 +0000 2026-02-26T03:25:58.077 INFO:teuthology.orchestra.run.trial012.stdout:Change: 2026-02-26 03:21:15.321054664 +0000 2026-02-26T03:25:58.077 INFO:teuthology.orchestra.run.trial012.stdout: Birth: 2026-02-26 03:21:15.321054664 +0000 2026-02-26T03:25:58.077 DEBUG:teuthology.orchestra.run.trial012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-26T03:25:58.141 INFO:teuthology.orchestra.run.trial012.stderr:1+0 records in 2026-02-26T03:25:58.141 INFO:teuthology.orchestra.run.trial012.stderr:1+0 records out 2026-02-26T03:25:58.141 INFO:teuthology.orchestra.run.trial012.stderr:512 bytes copied, 0.000174294 s, 2.9 MB/s 2026-02-26T03:25:58.142 DEBUG:teuthology.orchestra.run.trial012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-26T03:25:58.200 DEBUG:teuthology.orchestra.run.trial012:> stat /dev/vg_nvme/lv_2 2026-02-26T03:25:58.255 INFO:teuthology.orchestra.run.trial012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-26T03:25:58.255 INFO:teuthology.orchestra.run.trial012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:25:58.256 INFO:teuthology.orchestra.run.trial012.stdout:Device: 6h/6d Inode: 1288 Links: 1 2026-02-26T03:25:58.256 INFO:teuthology.orchestra.run.trial012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:25:58.256 INFO:teuthology.orchestra.run.trial012.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:25:58.256 INFO:teuthology.orchestra.run.trial012.stdout:Access: 2026-02-26 03:22:02.991678904 +0000 2026-02-26T03:25:58.256 INFO:teuthology.orchestra.run.trial012.stdout:Modify: 2026-02-26 03:21:15.320054693 +0000 2026-02-26T03:25:58.256 INFO:teuthology.orchestra.run.trial012.stdout:Change: 2026-02-26 03:21:15.320054693 +0000 2026-02-26T03:25:58.256 INFO:teuthology.orchestra.run.trial012.stdout: Birth: 2026-02-26 03:21:15.320054693 +0000 2026-02-26T03:25:58.256 DEBUG:teuthology.orchestra.run.trial012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-26T03:25:58.318 INFO:teuthology.orchestra.run.trial012.stderr:1+0 records in 2026-02-26T03:25:58.318 INFO:teuthology.orchestra.run.trial012.stderr:1+0 records out 2026-02-26T03:25:58.318 INFO:teuthology.orchestra.run.trial012.stderr:512 bytes copied, 0.000136984 s, 3.7 MB/s 2026-02-26T03:25:58.319 DEBUG:teuthology.orchestra.run.trial012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-26T03:25:58.377 DEBUG:teuthology.orchestra.run.trial012:> stat /dev/vg_nvme/lv_3 2026-02-26T03:25:58.432 INFO:teuthology.orchestra.run.trial012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-26T03:25:58.433 INFO:teuthology.orchestra.run.trial012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:25:58.433 INFO:teuthology.orchestra.run.trial012.stdout:Device: 6h/6d Inode: 1307 Links: 1 2026-02-26T03:25:58.433 INFO:teuthology.orchestra.run.trial012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:25:58.433 INFO:teuthology.orchestra.run.trial012.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:25:58.433 INFO:teuthology.orchestra.run.trial012.stdout:Access: 2026-02-26 03:22:02.992678875 +0000 2026-02-26T03:25:58.433 INFO:teuthology.orchestra.run.trial012.stdout:Modify: 2026-02-26 03:21:15.321054664 +0000 2026-02-26T03:25:58.433 INFO:teuthology.orchestra.run.trial012.stdout:Change: 2026-02-26 03:21:15.321054664 +0000 2026-02-26T03:25:58.433 INFO:teuthology.orchestra.run.trial012.stdout: Birth: 2026-02-26 03:21:15.321054664 +0000 2026-02-26T03:25:58.434 DEBUG:teuthology.orchestra.run.trial012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-26T03:25:58.495 INFO:teuthology.orchestra.run.trial012.stderr:1+0 records in 2026-02-26T03:25:58.495 INFO:teuthology.orchestra.run.trial012.stderr:1+0 records out 2026-02-26T03:25:58.495 INFO:teuthology.orchestra.run.trial012.stderr:512 bytes copied, 0.000149358 s, 3.4 MB/s 2026-02-26T03:25:58.496 DEBUG:teuthology.orchestra.run.trial012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-26T03:25:58.553 DEBUG:teuthology.orchestra.run.trial012:> stat /dev/vg_nvme/lv_4 2026-02-26T03:25:58.610 INFO:teuthology.orchestra.run.trial012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-26T03:25:58.610 INFO:teuthology.orchestra.run.trial012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:25:58.610 INFO:teuthology.orchestra.run.trial012.stdout:Device: 6h/6d Inode: 1306 Links: 1 2026-02-26T03:25:58.610 INFO:teuthology.orchestra.run.trial012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:25:58.610 INFO:teuthology.orchestra.run.trial012.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:25:58.611 INFO:teuthology.orchestra.run.trial012.stdout:Access: 2026-02-26 03:22:02.992678875 +0000 2026-02-26T03:25:58.611 INFO:teuthology.orchestra.run.trial012.stdout:Modify: 2026-02-26 03:21:15.321054664 +0000 2026-02-26T03:25:58.611 INFO:teuthology.orchestra.run.trial012.stdout:Change: 2026-02-26 03:21:15.321054664 +0000 2026-02-26T03:25:58.611 INFO:teuthology.orchestra.run.trial012.stdout: Birth: 2026-02-26 03:21:15.321054664 +0000 2026-02-26T03:25:58.611 DEBUG:teuthology.orchestra.run.trial012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-26T03:25:58.673 INFO:teuthology.orchestra.run.trial012.stderr:1+0 records in 2026-02-26T03:25:58.673 INFO:teuthology.orchestra.run.trial012.stderr:1+0 records out 2026-02-26T03:25:58.673 INFO:teuthology.orchestra.run.trial012.stderr:512 bytes copied, 0.000126625 s, 4.0 MB/s 2026-02-26T03:25:58.674 DEBUG:teuthology.orchestra.run.trial012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-26T03:25:58.732 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-26T03:25:58.732 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:25:58.732 DEBUG:teuthology.orchestra.run.trial068:> dd if=/scratch_devs of=/dev/stdout 2026-02-26T03:25:58.750 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-26T03:25:58.750 DEBUG:teuthology.orchestra.run.trial068:> stat /dev/vg_nvme/lv_1 2026-02-26T03:25:58.806 INFO:teuthology.orchestra.run.trial068.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-26T03:25:58.806 INFO:teuthology.orchestra.run.trial068.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:25:58.806 INFO:teuthology.orchestra.run.trial068.stdout:Device: 6h/6d Inode: 1299 Links: 1 2026-02-26T03:25:58.806 INFO:teuthology.orchestra.run.trial068.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:25:58.806 INFO:teuthology.orchestra.run.trial068.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:25:58.806 INFO:teuthology.orchestra.run.trial068.stdout:Access: 2026-02-26 03:25:31.951469186 +0000 2026-02-26T03:25:58.807 INFO:teuthology.orchestra.run.trial068.stdout:Modify: 2026-02-26 03:24:44.810633320 +0000 2026-02-26T03:25:58.807 INFO:teuthology.orchestra.run.trial068.stdout:Change: 2026-02-26 03:24:44.810633320 +0000 2026-02-26T03:25:58.807 INFO:teuthology.orchestra.run.trial068.stdout: Birth: 2026-02-26 03:24:44.810633320 +0000 2026-02-26T03:25:58.807 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-26T03:25:58.870 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records in 2026-02-26T03:25:58.870 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records out 2026-02-26T03:25:58.870 INFO:teuthology.orchestra.run.trial068.stderr:512 bytes copied, 0.000165721 s, 3.1 MB/s 2026-02-26T03:25:58.871 DEBUG:teuthology.orchestra.run.trial068:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-26T03:25:58.928 DEBUG:teuthology.orchestra.run.trial068:> stat /dev/vg_nvme/lv_2 2026-02-26T03:25:58.985 INFO:teuthology.orchestra.run.trial068.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-26T03:25:58.985 INFO:teuthology.orchestra.run.trial068.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:25:58.985 INFO:teuthology.orchestra.run.trial068.stdout:Device: 6h/6d Inode: 1291 Links: 1 2026-02-26T03:25:58.985 INFO:teuthology.orchestra.run.trial068.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:25:58.985 INFO:teuthology.orchestra.run.trial068.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:25:58.986 INFO:teuthology.orchestra.run.trial068.stdout:Access: 2026-02-26 03:25:31.952469183 +0000 2026-02-26T03:25:58.986 INFO:teuthology.orchestra.run.trial068.stdout:Modify: 2026-02-26 03:24:44.810633320 +0000 2026-02-26T03:25:58.986 INFO:teuthology.orchestra.run.trial068.stdout:Change: 2026-02-26 03:24:44.810633320 +0000 2026-02-26T03:25:58.986 INFO:teuthology.orchestra.run.trial068.stdout: Birth: 2026-02-26 03:24:44.810633320 +0000 2026-02-26T03:25:58.986 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-26T03:25:59.048 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records in 2026-02-26T03:25:59.048 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records out 2026-02-26T03:25:59.048 INFO:teuthology.orchestra.run.trial068.stderr:512 bytes copied, 0.000130165 s, 3.9 MB/s 2026-02-26T03:25:59.050 DEBUG:teuthology.orchestra.run.trial068:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-26T03:25:59.106 DEBUG:teuthology.orchestra.run.trial068:> stat /dev/vg_nvme/lv_3 2026-02-26T03:25:59.161 INFO:teuthology.orchestra.run.trial068.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-26T03:25:59.161 INFO:teuthology.orchestra.run.trial068.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:25:59.161 INFO:teuthology.orchestra.run.trial068.stdout:Device: 6h/6d Inode: 1303 Links: 1 2026-02-26T03:25:59.161 INFO:teuthology.orchestra.run.trial068.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:25:59.161 INFO:teuthology.orchestra.run.trial068.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:25:59.161 INFO:teuthology.orchestra.run.trial068.stdout:Access: 2026-02-26 03:25:31.952469183 +0000 2026-02-26T03:25:59.161 INFO:teuthology.orchestra.run.trial068.stdout:Modify: 2026-02-26 03:24:44.811633317 +0000 2026-02-26T03:25:59.161 INFO:teuthology.orchestra.run.trial068.stdout:Change: 2026-02-26 03:24:44.811633317 +0000 2026-02-26T03:25:59.161 INFO:teuthology.orchestra.run.trial068.stdout: Birth: 2026-02-26 03:24:44.811633317 +0000 2026-02-26T03:25:59.162 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-26T03:25:59.224 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records in 2026-02-26T03:25:59.224 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records out 2026-02-26T03:25:59.224 INFO:teuthology.orchestra.run.trial068.stderr:512 bytes copied, 0.000224251 s, 2.3 MB/s 2026-02-26T03:25:59.225 DEBUG:teuthology.orchestra.run.trial068:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-26T03:25:59.281 DEBUG:teuthology.orchestra.run.trial068:> stat /dev/vg_nvme/lv_4 2026-02-26T03:25:59.337 INFO:teuthology.orchestra.run.trial068.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-26T03:25:59.337 INFO:teuthology.orchestra.run.trial068.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:25:59.337 INFO:teuthology.orchestra.run.trial068.stdout:Device: 6h/6d Inode: 1311 Links: 1 2026-02-26T03:25:59.337 INFO:teuthology.orchestra.run.trial068.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:25:59.337 INFO:teuthology.orchestra.run.trial068.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:25:59.337 INFO:teuthology.orchestra.run.trial068.stdout:Access: 2026-02-26 03:25:31.952469183 +0000 2026-02-26T03:25:59.337 INFO:teuthology.orchestra.run.trial068.stdout:Modify: 2026-02-26 03:24:44.811633317 +0000 2026-02-26T03:25:59.337 INFO:teuthology.orchestra.run.trial068.stdout:Change: 2026-02-26 03:24:44.811633317 +0000 2026-02-26T03:25:59.337 INFO:teuthology.orchestra.run.trial068.stdout: Birth: 2026-02-26 03:24:44.811633317 +0000 2026-02-26T03:25:59.337 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-26T03:25:59.400 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records in 2026-02-26T03:25:59.400 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records out 2026-02-26T03:25:59.400 INFO:teuthology.orchestra.run.trial068.stderr:512 bytes copied, 0.000209995 s, 2.4 MB/s 2026-02-26T03:25:59.401 DEBUG:teuthology.orchestra.run.trial068:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-26T03:25:59.458 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-26T03:25:59.459 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial068.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-26T03:25:59.459 INFO:tasks.ceph:Generating config... 2026-02-26T03:25:59.461 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-26T03:25:59.461 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-26T03:25:59.461 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-26T03:25:59.462 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-26T03:25:59.462 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-26T03:25:59.462 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-26T03:25:59.462 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-26T03:25:59.462 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-26T03:25:59.462 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-26T03:25:59.462 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-26T03:25:59.462 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-26T03:25:59.462 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-26T03:25:59.462 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-26T03:25:59.462 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-26T03:25:59.462 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-26T03:25:59.463 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-26T03:25:59.464 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-26T03:25:59.464 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-26T03:25:59.464 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-26T03:25:59.464 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-26T03:25:59.464 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-26T03:25:59.464 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-26T03:25:59.464 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-26T03:25:59.464 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-26T03:25:59.464 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-26T03:25:59.464 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-26T03:25:59.464 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-26T03:25:59.464 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-26T03:25:59.464 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-26T03:25:59.465 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-26T03:25:59.465 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-26T03:25:59.465 INFO:tasks.ceph:Setting up mon.a... 2026-02-26T03:25:59.465 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-26T03:25:59.507 INFO:teuthology.orchestra.run.trial012.stdout:creating /etc/ceph/ceph.keyring 2026-02-26T03:25:59.510 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-26T03:25:59.592 DEBUG:teuthology.orchestra.run.trial012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-26T03:25:59.656 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.12')] 2026-02-26T03:25:59.657 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.12', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-26T03:25:59.659 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:25:59.659 DEBUG:teuthology.orchestra.run.trial012:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-26T03:25:59.713 DEBUG:teuthology.orchestra.run.trial012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.12 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:25:59.792 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:25:59.790+0000 7f1b30033f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:25:59.793 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:25:59.790+0000 7f1b30033f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:25:59.793 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:25:59.791+0000 7f1b30033f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:25:59.793 INFO:teuthology.orchestra.run.trial012.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:25:59.793 INFO:teuthology.orchestra.run.trial012.stdout:monmaptool: generated fsid dc070458-d616-4479-8618-4ead68863657 2026-02-26T03:25:59.794 INFO:teuthology.orchestra.run.trial012.stdout:setting min_mon_release = tentacle 2026-02-26T03:25:59.794 INFO:teuthology.orchestra.run.trial012.stdout:epoch 0 2026-02-26T03:25:59.794 INFO:teuthology.orchestra.run.trial012.stdout:fsid dc070458-d616-4479-8618-4ead68863657 2026-02-26T03:25:59.794 INFO:teuthology.orchestra.run.trial012.stdout:last_changed 2026-02-26T03:25:59.791896+0000 2026-02-26T03:25:59.794 INFO:teuthology.orchestra.run.trial012.stdout:created 2026-02-26T03:25:59.791896+0000 2026-02-26T03:25:59.794 INFO:teuthology.orchestra.run.trial012.stdout:min_mon_release 20 (tentacle) 2026-02-26T03:25:59.794 INFO:teuthology.orchestra.run.trial012.stdout:election_strategy: 1 2026-02-26T03:25:59.794 INFO:teuthology.orchestra.run.trial012.stdout:0: [v2:10.20.193.12:3300/0,v1:10.20.193.12:6789/0] mon.a 2026-02-26T03:25:59.794 INFO:teuthology.orchestra.run.trial012.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-26T03:25:59.796 DEBUG:teuthology.orchestra.run.trial012:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-26T03:25:59.852 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID dc070458-d616-4479-8618-4ead68863657... 2026-02-26T03:25:59.853 DEBUG:teuthology.orchestra.run.trial012:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-26T03:25:59.894 DEBUG:teuthology.orchestra.run.trial068:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-26T03:25:59.930 INFO:teuthology.orchestra.run.trial012.stdout:[global] 2026-02-26T03:25:59.931 INFO:teuthology.orchestra.run.trial012.stdout: chdir = "" 2026-02-26T03:25:59.931 INFO:teuthology.orchestra.run.trial012.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-26T03:25:59.931 INFO:teuthology.orchestra.run.trial012.stdout: auth supported = cephx 2026-02-26T03:25:59.931 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.931 INFO:teuthology.orchestra.run.trial012.stdout: filestore xattr use omap = true 2026-02-26T03:25:59.931 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.931 INFO:teuthology.orchestra.run.trial012.stdout: mon clock drift allowed = 1.000 2026-02-26T03:25:59.931 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.931 INFO:teuthology.orchestra.run.trial012.stdout: osd crush chooseleaf type = 0 2026-02-26T03:25:59.931 INFO:teuthology.orchestra.run.trial012.stdout: auth debug = true 2026-02-26T03:25:59.932 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.932 INFO:teuthology.orchestra.run.trial012.stdout: ms die on old message = true 2026-02-26T03:25:59.932 INFO:teuthology.orchestra.run.trial012.stdout: ms die on bug = true 2026-02-26T03:25:59.932 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.932 INFO:teuthology.orchestra.run.trial012.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-26T03:25:59.932 INFO:teuthology.orchestra.run.trial012.stdout: mon pg warn max object skew = 0 2026-02-26T03:25:59.932 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.932 INFO:teuthology.orchestra.run.trial012.stdout: # disable pg_autoscaler by default for new pools 2026-02-26T03:25:59.932 INFO:teuthology.orchestra.run.trial012.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-26T03:25:59.932 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.932 INFO:teuthology.orchestra.run.trial012.stdout: osd pool default size = 2 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: mon osd allow primary affinity = true 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: mon osd allow pg remap = true 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: mon warn on legacy crush tunables = false 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: mon warn on crush straw calc version zero = false 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: mon warn on no sortbitwise = false 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: mon warn on osd down out interval zero = false 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: mon warn on too few osds = false 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: mon_allow_pool_size_one = true 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.933 INFO:teuthology.orchestra.run.trial012.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: osd default data pool replay window = 5 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: mon allow pool delete = true 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: mon cluster log file level = debug 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: debug asserts on shutdown = true 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: mon health detail to clog = false 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: mon host = 10.20.193.12 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: ms client mode = crc 2026-02-26T03:25:59.934 INFO:teuthology.orchestra.run.trial012.stdout: ms cluster mode = crc 2026-02-26T03:25:59.935 INFO:teuthology.orchestra.run.trial012.stdout: ms mon client mode = crc 2026-02-26T03:25:59.935 INFO:teuthology.orchestra.run.trial012.stdout: ms mon cluster mode = crc 2026-02-26T03:25:59.935 INFO:teuthology.orchestra.run.trial012.stdout: ms mon service mode = crc 2026-02-26T03:25:59.935 INFO:teuthology.orchestra.run.trial012.stdout: ms service mode = crc 2026-02-26T03:25:59.935 INFO:teuthology.orchestra.run.trial012.stdout: fsid = dc070458-d616-4479-8618-4ead68863657 2026-02-26T03:25:59.935 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.935 INFO:teuthology.orchestra.run.trial012.stdout:[osd] 2026-02-26T03:25:59.935 INFO:teuthology.orchestra.run.trial012.stdout: osd journal size = 100 2026-02-26T03:25:59.935 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.935 INFO:teuthology.orchestra.run.trial012.stdout: osd scrub load threshold = 5.0 2026-02-26T03:25:59.935 INFO:teuthology.orchestra.run.trial012.stdout: osd scrub max interval = 600 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: osd mclock profile = high_recovery_ops 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: osd mclock skip benchmark = true 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: osd recover clone overlap = true 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: osd recovery max chunk = 1048576 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: osd debug shutdown = true 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: osd debug op order = true 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: osd debug verify stray on activate = true 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: osd debug trim objects = true 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: osd open classes on start = true 2026-02-26T03:25:59.936 INFO:teuthology.orchestra.run.trial012.stdout: osd debug pg log writeout = true 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: osd deep scrub update digest min age = 30 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: osd map max advance = 10 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: journal zero on create = true 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: filestore ondisk finisher threads = 3 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: filestore apply finisher threads = 3 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: bdev debug aio = true 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: osd debug misdirected ops = true 2026-02-26T03:25:59.937 INFO:teuthology.orchestra.run.trial012.stdout: debug monc = 20 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: crimson cpu num = 4 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: crimson osd obc lru size = 10 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: debug ms = 20 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: debug osd = 20 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore = 20 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore backref = 20 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore cache = 20 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore cleaner = 20 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore device = 20 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore epm = 20 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore fixedkv tree = 20 2026-02-26T03:25:59.938 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore journal = 20 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore lba = 20 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore odata = 20 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore omap = 20 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore onode = 20 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore t = 20 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: debug seastore tm = 20 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: osd objectstore = seastore 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: osd shutdown pgref assert = True 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: seastore cachepin size pershard = 64M 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: seastore cachepin type = 2Q 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: seastore max concurrent transactions = 8 2026-02-26T03:25:59.939 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout:[mgr] 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout: debug ms = 1 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout: debug mgr = 20 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout: debug mon = 20 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout: debug auth = 20 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout: mon reweight min pgs per osd = 4 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout: mon reweight min bytes per osd = 10 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout: mgr/telemetry/nag = false 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout:[mon] 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout: debug ms = 1 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout: debug mon = 20 2026-02-26T03:25:59.940 INFO:teuthology.orchestra.run.trial012.stdout: debug paxos = 20 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: debug auth = 20 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: mon data avail warn = 5 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: mon mgr mkfs grace = 240 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: mon reweight min pgs per osd = 4 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: mon osd reporter subtree level = osd 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: mon osd prime pg temp = true 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: mon reweight min bytes per osd = 10 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: auth mon ticket ttl = 660 # 11m 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: auth service ticket ttl = 240 # 4m 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.941 INFO:teuthology.orchestra.run.trial012.stdout: # don't complain about insecure global_id in the test suite 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: # 1m isn't quite enough 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: mon_down_mkfs_grace = 2m 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: mon_warn_on_filestore_osds = false 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: mon min osdmap epochs = 50 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: mon osdmap full prune interval = 2 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: mon osdmap full prune min = 15 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: mon osdmap full prune txsize = 2 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: paxos service trim min = 10 2026-02-26T03:25:59.942 INFO:teuthology.orchestra.run.trial012.stdout: osd pool default crimson = True 2026-02-26T03:25:59.943 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:25:59.943 INFO:teuthology.orchestra.run.trial012.stdout:[client] 2026-02-26T03:25:59.943 INFO:teuthology.orchestra.run.trial012.stdout: rgw cache enabled = true 2026-02-26T03:25:59.943 INFO:teuthology.orchestra.run.trial012.stdout: rgw enable ops log = true 2026-02-26T03:25:59.943 INFO:teuthology.orchestra.run.trial012.stdout: rgw enable usage log = true 2026-02-26T03:25:59.943 INFO:teuthology.orchestra.run.trial012.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-26T03:25:59.943 INFO:teuthology.orchestra.run.trial012.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-26T03:25:59.943 INFO:teuthology.orchestra.run.trial012.stdout:[mon.a] 2026-02-26T03:25:59.945 INFO:teuthology.orchestra.run.trial068.stdout:[global] 2026-02-26T03:25:59.945 INFO:teuthology.orchestra.run.trial068.stdout: chdir = "" 2026-02-26T03:25:59.945 INFO:teuthology.orchestra.run.trial068.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-26T03:25:59.945 INFO:teuthology.orchestra.run.trial068.stdout: auth supported = cephx 2026-02-26T03:25:59.945 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: filestore xattr use omap = true 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: mon clock drift allowed = 1.000 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: osd crush chooseleaf type = 0 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: auth debug = true 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: ms die on old message = true 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: ms die on bug = true 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-26T03:25:59.946 INFO:teuthology.orchestra.run.trial068.stdout: mon pg warn max object skew = 0 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: # disable pg_autoscaler by default for new pools 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: osd pool default size = 2 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: mon osd allow primary affinity = true 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: mon osd allow pg remap = true 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: mon warn on legacy crush tunables = false 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: mon warn on crush straw calc version zero = false 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: mon warn on no sortbitwise = false 2026-02-26T03:25:59.947 INFO:teuthology.orchestra.run.trial068.stdout: mon warn on osd down out interval zero = false 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: mon warn on too few osds = false 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: mon_allow_pool_size_one = true 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: osd default data pool replay window = 5 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: mon allow pool delete = true 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: mon cluster log file level = debug 2026-02-26T03:25:59.948 INFO:teuthology.orchestra.run.trial068.stdout: debug asserts on shutdown = true 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout: mon health detail to clog = false 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout: mon host = 10.20.193.12 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout: ms client mode = crc 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout: ms cluster mode = crc 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout: ms mon client mode = crc 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout: ms mon cluster mode = crc 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout: ms mon service mode = crc 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout: ms service mode = crc 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout: fsid = dc070458-d616-4479-8618-4ead68863657 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.949 INFO:teuthology.orchestra.run.trial068.stdout:[osd] 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: osd journal size = 100 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: osd scrub load threshold = 5.0 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: osd scrub max interval = 600 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: osd mclock profile = high_recovery_ops 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: osd mclock skip benchmark = true 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: osd recover clone overlap = true 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: osd recovery max chunk = 1048576 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: osd debug shutdown = true 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: osd debug op order = true 2026-02-26T03:25:59.950 INFO:teuthology.orchestra.run.trial068.stdout: osd debug verify stray on activate = true 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: osd debug trim objects = true 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: osd open classes on start = true 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: osd debug pg log writeout = true 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: osd deep scrub update digest min age = 30 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: osd map max advance = 10 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: journal zero on create = true 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.951 INFO:teuthology.orchestra.run.trial068.stdout: filestore ondisk finisher threads = 3 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: filestore apply finisher threads = 3 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: bdev debug aio = true 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: osd debug misdirected ops = true 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: debug monc = 20 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: crimson cpu num = 4 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: crimson osd obc lru size = 10 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: debug ms = 20 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: debug osd = 20 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore = 20 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore backref = 20 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore cache = 20 2026-02-26T03:25:59.952 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore cleaner = 20 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore device = 20 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore epm = 20 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore fixedkv tree = 20 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore journal = 20 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore lba = 20 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore odata = 20 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore omap = 20 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore onode = 20 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore t = 20 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: debug seastore tm = 20 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: osd objectstore = seastore 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: osd shutdown pgref assert = True 2026-02-26T03:25:59.953 INFO:teuthology.orchestra.run.trial068.stdout: seastore cachepin size pershard = 64M 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout: seastore cachepin type = 2Q 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout: seastore max concurrent transactions = 8 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout:[mgr] 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout: debug ms = 1 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout: debug mgr = 20 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout: debug mon = 20 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout: debug auth = 20 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout: mon reweight min pgs per osd = 4 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout: mon reweight min bytes per osd = 10 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout: mgr/telemetry/nag = false 2026-02-26T03:25:59.954 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout:[mon] 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: debug ms = 1 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: debug mon = 20 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: debug paxos = 20 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: debug auth = 20 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: mon data avail warn = 5 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: mon mgr mkfs grace = 240 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: mon reweight min pgs per osd = 4 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: mon osd reporter subtree level = osd 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: mon osd prime pg temp = true 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: mon reweight min bytes per osd = 10 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.955 INFO:teuthology.orchestra.run.trial068.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: auth mon ticket ttl = 660 # 11m 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: auth service ticket ttl = 240 # 4m 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: # don't complain about insecure global_id in the test suite 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: # 1m isn't quite enough 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: mon_down_mkfs_grace = 2m 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: mon_warn_on_filestore_osds = false 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: mon min osdmap epochs = 50 2026-02-26T03:25:59.956 INFO:teuthology.orchestra.run.trial068.stdout: mon osdmap full prune interval = 2 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout: mon osdmap full prune min = 15 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout: mon osdmap full prune txsize = 2 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout: paxos service trim min = 10 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout: osd pool default crimson = True 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout:[client] 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout: rgw cache enabled = true 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout: rgw enable ops log = true 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout: rgw enable usage log = true 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-26T03:25:59.957 INFO:teuthology.orchestra.run.trial068.stdout:[mon.a] 2026-02-26T03:25:59.958 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-26T03:25:59.958 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-26T03:26:00.035 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-26T03:26:00.036 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:26:00.036 DEBUG:teuthology.orchestra.run.trial012:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-26T03:26:00.052 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:26:00.052 DEBUG:teuthology.orchestra.run.trial012:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-26T03:26:00.109 INFO:tasks.ceph:Sending monmap to node ubuntu@trial012.front.sepia.ceph.com 2026-02-26T03:26:00.109 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:26:00.110 DEBUG:teuthology.orchestra.run.trial012:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-26T03:26:00.110 DEBUG:teuthology.orchestra.run.trial012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-26T03:26:00.183 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:26:00.183 DEBUG:teuthology.orchestra.run.trial012:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:26:00.240 INFO:tasks.ceph:Sending monmap to node ubuntu@trial068.front.sepia.ceph.com 2026-02-26T03:26:00.241 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:26:00.241 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-26T03:26:00.241 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-26T03:26:00.274 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:26:00.275 DEBUG:teuthology.orchestra.run.trial068:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:26:00.330 INFO:tasks.ceph:Setting up mon nodes... 2026-02-26T03:26:00.330 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-26T03:26:00.330 DEBUG:teuthology.orchestra.run.trial068:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-26T03:26:00.420 INFO:teuthology.orchestra.run.trial068.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-26T03:26:00.424 INFO:tasks.ceph:Setting up mds nodes... 2026-02-26T03:26:00.424 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-26T03:26:00.424 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-26T03:26:00.466 INFO:teuthology.orchestra.run.trial012.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-26T03:26:00.478 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2026-02-26T03:26:00.519 INFO:teuthology.orchestra.run.trial068.stdout:creating /etc/ceph/ceph.client.1.keyring 2026-02-26T03:26:00.531 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-26T03:26:00.531 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial068.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-26T03:26:00.531 DEBUG:teuthology.orchestra.run.trial012:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-26T03:26:00.556 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-26T03:26:00.556 INFO:tasks.ceph:role: osd.0 2026-02-26T03:26:00.557 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial012.front.sepia.ceph.com 2026-02-26T03:26:00.557 DEBUG:teuthology.orchestra.run.trial012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-26T03:26:00.621 INFO:teuthology.orchestra.run.trial012.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-26T03:26:00.622 INFO:teuthology.orchestra.run.trial012.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-26T03:26:00.622 INFO:teuthology.orchestra.run.trial012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-26T03:26:00.622 INFO:teuthology.orchestra.run.trial012.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-26T03:26:00.622 INFO:teuthology.orchestra.run.trial012.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-26T03:26:00.622 INFO:teuthology.orchestra.run.trial012.stdout: = sunit=0 swidth=0 blks 2026-02-26T03:26:00.622 INFO:teuthology.orchestra.run.trial012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-26T03:26:00.622 INFO:teuthology.orchestra.run.trial012.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-26T03:26:00.622 INFO:teuthology.orchestra.run.trial012.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-26T03:26:00.622 INFO:teuthology.orchestra.run.trial012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-26T03:26:01.187 INFO:teuthology.orchestra.run.trial012.stdout:Discarding blocks...Done. 2026-02-26T03:26:01.189 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial012.front.sepia.ceph.com -o noatime 2026-02-26T03:26:01.189 DEBUG:teuthology.orchestra.run.trial012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-26T03:26:01.258 DEBUG:teuthology.orchestra.run.trial012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-26T03:26:01.326 DEBUG:teuthology.orchestra.run.trial012:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-26T03:26:01.390 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-26T03:26:01.390 INFO:tasks.ceph:role: osd.1 2026-02-26T03:26:01.390 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial012.front.sepia.ceph.com 2026-02-26T03:26:01.391 DEBUG:teuthology.orchestra.run.trial012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-26T03:26:01.455 INFO:teuthology.orchestra.run.trial012.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-26T03:26:01.456 INFO:teuthology.orchestra.run.trial012.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-26T03:26:01.456 INFO:teuthology.orchestra.run.trial012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-26T03:26:01.456 INFO:teuthology.orchestra.run.trial012.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-26T03:26:01.456 INFO:teuthology.orchestra.run.trial012.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-26T03:26:01.456 INFO:teuthology.orchestra.run.trial012.stdout: = sunit=0 swidth=0 blks 2026-02-26T03:26:01.456 INFO:teuthology.orchestra.run.trial012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-26T03:26:01.456 INFO:teuthology.orchestra.run.trial012.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-26T03:26:01.456 INFO:teuthology.orchestra.run.trial012.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-26T03:26:01.456 INFO:teuthology.orchestra.run.trial012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-26T03:26:02.026 INFO:teuthology.orchestra.run.trial012.stdout:Discarding blocks...Done. 2026-02-26T03:26:02.028 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial012.front.sepia.ceph.com -o noatime 2026-02-26T03:26:02.029 DEBUG:teuthology.orchestra.run.trial012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-26T03:26:02.097 DEBUG:teuthology.orchestra.run.trial012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-26T03:26:02.164 DEBUG:teuthology.orchestra.run.trial012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:26:02.314 INFO:teuthology.orchestra.run.trial012.stderr:INFO  osd - OSD::main: parsing early config 2026-02-26T03:26:02.376 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:02,375 seastar - Reactor backend: linux-aio 2026-02-26T03:26:02.393 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:02,392 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:02.393 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:02,392 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:02.395 INFO:teuthology.orchestra.run.trial012.stderr:==37262==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:26:02.449 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:02,448 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-26T03:26:02.450 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:02,448 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-26T03:26:02.450 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:02,448 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-26T03:26:02.450 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:02,448 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-26T03:26:02.450 INFO:teuthology.orchestra.run.trial012.stderr:DEBUG 2026-02-26 03:26:02,448 [shard 0:main] seastar - reactor::drain 2026-02-26T03:26:04.488 INFO:teuthology.orchestra.run.trial012.stderr:----------------------------------------------------- 2026-02-26T03:26:04.488 INFO:teuthology.orchestra.run.trial012.stderr:Suppressions used: 2026-02-26T03:26:04.488 INFO:teuthology.orchestra.run.trial012.stderr: count bytes template 2026-02-26T03:26:04.488 INFO:teuthology.orchestra.run.trial012.stderr: 1 8 MallocExtension::Register 2026-02-26T03:26:04.489 INFO:teuthology.orchestra.run.trial012.stderr:----------------------------------------------------- 2026-02-26T03:26:04.489 INFO:teuthology.orchestra.run.trial012.stderr: 2026-02-26T03:26:04.507 INFO:teuthology.orchestra.run.trial012.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-26T03:26:04.507 INFO:teuthology.orchestra.run.trial012.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-26T03:26:04.555 INFO:teuthology.orchestra.run.trial012.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-26T03:26:04.555 INFO:teuthology.orchestra.run.trial012.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-26T03:26:04.562 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:04,561 seastar - Reactor backend: linux-aio 2026-02-26T03:26:04.576 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:04,575 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:04.576 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:04,575 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:04.580 INFO:teuthology.orchestra.run.trial012.stderr:==37260==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:26:04.581 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:04,579 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-26T03:26:04.638 INFO:teuthology.orchestra.run.trial012.stderr:DEBUG 2026-02-26 03:26:04,637 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-26T03:26:04.719 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb052f /lib64/libasan.so.8+0xb1437 0xb69757 0x1090878 0xf2e0dc 0x9c54dff 0x9d4998d 0x78947dd 0x7bea228 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:04.719 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xfd19a /lib64/libasan.so.8+0xec7e9 0xcb04f1 0x7bea68a 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:04.719 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x9c98905 0x9d47a94 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:04.750 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0xe0b9b9 0x7e5dc77 0x7e64490 0x8f5d61d 0x8f5ff8b 0x787cd7f 0x78b3782 0x6fa18b5 0x755a4ec 0x755a640 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f7a738 0x9bf83c1 0x9bf8bfc 0xecf14d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46f44 2026-02-26T03:26:04.816 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfcaf3 /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f7a738 0x9bf83c1 0x9bf8bfc 0xecf14d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46f44 2026-02-26T03:26:04.827 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:04.854 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x2a00b9c 0x2a559e0 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:04.855 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:04.908 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 33 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x1125428 0xf2e0d1 0x9c50c5c 0x9c50f59 0x9c511cc 0x9c521cc 0x9c5a24f 0x9c5a5cb 0x9d269d4 0x9d2737f 0x9d2787d 0x9d27b70 0x9c076fb 0x9c07936 0x9c07c2d 0x9c07caa 0x7b75d17 0x7b763e2 0x79615c3 0x79a8640 0x7a800ed 0x7a802a6 0x72b17b9 0x72b7733 0x6f92f40 0x755a74a 0x755a89e 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:04.982 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x1125423 0xf2e0d1 0x9c50a8f 0x9c50f59 0x9c511cc 0x9c51482 0x9c51a44 0x9c59eeb 0x9c5a5cb 0x9d43118 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:04.982 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65d8 /lib64/libasan.so.8+0x15197 0x9c0f995 0x9d257eb 0x9d25a92 0x9d25bfb 0x9d25d5c 0x9d26250 0x9d42adb 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:04.982 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x10908ef 0xf2e0dc 0x9c50c5c 0x9c50f59 0x9c511cc 0x9c51482 0x9c51f31 0x9c5242f 0x9c52a5f 0x9c52bc9 0x9d32f39 0x9d45d3c 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:05.064 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x2a00bea 0x2a559e0 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f7a738 0x9bf83c1 0x9bf8bfc 0xecf14d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46f44 2026-02-26T03:26:05.078 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x2a553df 0x7d837f4 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:05.079 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x2a54d9d 0x7d837e7 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:05.079 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:07.239 INFO:teuthology.orchestra.run.trial012.stderr:----------------------------------------------------- 2026-02-26T03:26:07.239 INFO:teuthology.orchestra.run.trial012.stderr:Suppressions used: 2026-02-26T03:26:07.239 INFO:teuthology.orchestra.run.trial012.stderr: count bytes template 2026-02-26T03:26:07.240 INFO:teuthology.orchestra.run.trial012.stderr: 1 8 MallocExtension::Register 2026-02-26T03:26:07.240 INFO:teuthology.orchestra.run.trial012.stderr:----------------------------------------------------- 2026-02-26T03:26:07.240 INFO:teuthology.orchestra.run.trial012.stderr: 2026-02-26T03:26:07.273 DEBUG:teuthology.orchestra.run.trial012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-26T03:26:07.297 DEBUG:teuthology.orchestra.run.trial012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:26:07.449 INFO:teuthology.orchestra.run.trial012.stderr:INFO  osd - OSD::main: parsing early config 2026-02-26T03:26:07.507 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:07,505 seastar - Reactor backend: linux-aio 2026-02-26T03:26:07.516 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:07,515 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:07.516 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:07,515 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:07.518 INFO:teuthology.orchestra.run.trial012.stderr:==37325==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:26:07.574 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:07,572 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-26T03:26:07.574 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:07,572 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-26T03:26:07.574 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:07,572 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-26T03:26:07.574 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:07,572 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-26T03:26:07.574 INFO:teuthology.orchestra.run.trial012.stderr:DEBUG 2026-02-26 03:26:07,573 [shard 0:main] seastar - reactor::drain 2026-02-26T03:26:09.575 INFO:teuthology.orchestra.run.trial012.stderr:----------------------------------------------------- 2026-02-26T03:26:09.575 INFO:teuthology.orchestra.run.trial012.stderr:Suppressions used: 2026-02-26T03:26:09.575 INFO:teuthology.orchestra.run.trial012.stderr: count bytes template 2026-02-26T03:26:09.575 INFO:teuthology.orchestra.run.trial012.stderr: 1 8 MallocExtension::Register 2026-02-26T03:26:09.575 INFO:teuthology.orchestra.run.trial012.stderr:----------------------------------------------------- 2026-02-26T03:26:09.575 INFO:teuthology.orchestra.run.trial012.stderr: 2026-02-26T03:26:09.594 INFO:teuthology.orchestra.run.trial012.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-26T03:26:09.594 INFO:teuthology.orchestra.run.trial012.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-26T03:26:09.642 INFO:teuthology.orchestra.run.trial012.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-26T03:26:09.642 INFO:teuthology.orchestra.run.trial012.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-26T03:26:09.649 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:09,648 seastar - Reactor backend: linux-aio 2026-02-26T03:26:09.663 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:09,662 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:09.663 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:09,662 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:09.667 INFO:teuthology.orchestra.run.trial012.stderr:==37323==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:26:09.667 INFO:teuthology.orchestra.run.trial012.stderr:INFO 2026-02-26 03:26:09,666 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-26T03:26:09.725 INFO:teuthology.orchestra.run.trial012.stderr:DEBUG 2026-02-26 03:26:09,724 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-26T03:26:09.805 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6583 /lib64/libasan.so.8+0x15197 0x1125428 0xf2e0d1 0x9c50a8f 0x9c50f59 0x9c511cc 0x9c51482 0x9c51f31 0x9c5242f 0x9c52a5f 0x9c52bc9 0x9d32f39 0x9d45d3c 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:09.805 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15876 0x9c544de 0x9c550b9 0x9d3f001 0x9d496e7 0x78947dd 0x7bea228 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:09.805 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x9c513a8 0x9c51a44 0x9c59eeb 0x9c5a5cb 0x9d43118 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:09.902 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f7a738 0x9bf83c1 0x9bf8bfc 0xecf14d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46f44 2026-02-26T03:26:09.942 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:09.943 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfcafa /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:09.946 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x2a00bbf 0x2a559e0 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:10.068 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xeb382 0xb69825 0xb75aef 0xbb92ea 0xe05f80 0x7c933ed 0x7bdaa69 0x7be9ecd 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:10.068 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15eac 0x9d385ff 0x9d38b76 0x9d390bc 0x9d4334a 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:10.068 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe1d59b 0x9ba45bf 0x9d1b824 0x9d1b91d 0x7bea265 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:10.068 INFO:teuthology.orchestra.run.trial012.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff8301ff44 0xffffffff830203b0 0xffffffff82e9525e 0xffffffff831e7269 0xffffffff831ca7d5 0xffffffff831d4bab 0xffffffff831d500b 0xffffffff82e910d2 0xffffffff83b50451 0xffffffff83c00ba2 2026-02-26T03:26:10.150 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xf4e9a /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f7a738 0x9bf83c1 0x9bf8bfc 0xecf14d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46f44 2026-02-26T03:26:10.165 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:10.165 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:10.165 INFO:teuthology.orchestra.run.trial012.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ec6 /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:12.350 INFO:teuthology.orchestra.run.trial012.stderr:----------------------------------------------------- 2026-02-26T03:26:12.350 INFO:teuthology.orchestra.run.trial012.stderr:Suppressions used: 2026-02-26T03:26:12.350 INFO:teuthology.orchestra.run.trial012.stderr: count bytes template 2026-02-26T03:26:12.350 INFO:teuthology.orchestra.run.trial012.stderr: 1 8 MallocExtension::Register 2026-02-26T03:26:12.350 INFO:teuthology.orchestra.run.trial012.stderr:----------------------------------------------------- 2026-02-26T03:26:12.350 INFO:teuthology.orchestra.run.trial012.stderr: 2026-02-26T03:26:12.385 DEBUG:teuthology.orchestra.run.trial012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-26T03:26:12.411 DEBUG:teuthology.orchestra.run.trial068:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-26T03:26:12.437 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-26T03:26:12.438 INFO:tasks.ceph:role: osd.2 2026-02-26T03:26:12.438 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial068.front.sepia.ceph.com 2026-02-26T03:26:12.438 DEBUG:teuthology.orchestra.run.trial068:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-26T03:26:12.503 INFO:teuthology.orchestra.run.trial068.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-26T03:26:12.504 INFO:teuthology.orchestra.run.trial068.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-26T03:26:12.504 INFO:teuthology.orchestra.run.trial068.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-26T03:26:12.504 INFO:teuthology.orchestra.run.trial068.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-26T03:26:12.504 INFO:teuthology.orchestra.run.trial068.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-26T03:26:12.504 INFO:teuthology.orchestra.run.trial068.stdout: = sunit=0 swidth=0 blks 2026-02-26T03:26:12.504 INFO:teuthology.orchestra.run.trial068.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-26T03:26:12.504 INFO:teuthology.orchestra.run.trial068.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-26T03:26:12.504 INFO:teuthology.orchestra.run.trial068.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-26T03:26:12.504 INFO:teuthology.orchestra.run.trial068.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-26T03:26:13.077 INFO:teuthology.orchestra.run.trial068.stdout:Discarding blocks...Done. 2026-02-26T03:26:13.079 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial068.front.sepia.ceph.com -o noatime 2026-02-26T03:26:13.079 DEBUG:teuthology.orchestra.run.trial068:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2026-02-26T03:26:13.151 DEBUG:teuthology.orchestra.run.trial068:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-26T03:26:13.219 DEBUG:teuthology.orchestra.run.trial068:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-26T03:26:13.284 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-26T03:26:13.284 INFO:tasks.ceph:role: osd.3 2026-02-26T03:26:13.284 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial068.front.sepia.ceph.com 2026-02-26T03:26:13.284 DEBUG:teuthology.orchestra.run.trial068:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-26T03:26:13.348 INFO:teuthology.orchestra.run.trial068.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-26T03:26:13.348 INFO:teuthology.orchestra.run.trial068.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-26T03:26:13.349 INFO:teuthology.orchestra.run.trial068.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-26T03:26:13.349 INFO:teuthology.orchestra.run.trial068.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-26T03:26:13.349 INFO:teuthology.orchestra.run.trial068.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-26T03:26:13.349 INFO:teuthology.orchestra.run.trial068.stdout: = sunit=0 swidth=0 blks 2026-02-26T03:26:13.349 INFO:teuthology.orchestra.run.trial068.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-26T03:26:13.349 INFO:teuthology.orchestra.run.trial068.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-26T03:26:13.349 INFO:teuthology.orchestra.run.trial068.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-26T03:26:13.349 INFO:teuthology.orchestra.run.trial068.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-26T03:26:13.911 INFO:teuthology.orchestra.run.trial068.stdout:Discarding blocks...Done. 2026-02-26T03:26:13.913 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial068.front.sepia.ceph.com -o noatime 2026-02-26T03:26:13.913 DEBUG:teuthology.orchestra.run.trial068:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2026-02-26T03:26:13.981 DEBUG:teuthology.orchestra.run.trial068:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-26T03:26:14.047 DEBUG:teuthology.orchestra.run.trial068:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:26:14.199 INFO:teuthology.orchestra.run.trial068.stderr:INFO  osd - OSD::main: parsing early config 2026-02-26T03:26:14.252 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:14,248 seastar - Reactor backend: linux-aio 2026-02-26T03:26:14.261 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:14,258 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:14.261 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:14,258 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:14.263 INFO:teuthology.orchestra.run.trial068.stderr:==36501==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:26:14.318 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:14,315 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-26T03:26:14.318 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:14,315 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-26T03:26:14.319 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:14,315 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-26T03:26:14.319 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:14,315 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-26T03:26:14.319 INFO:teuthology.orchestra.run.trial068.stderr:DEBUG 2026-02-26 03:26:14,315 [shard 0:main] seastar - reactor::drain 2026-02-26T03:26:16.368 INFO:teuthology.orchestra.run.trial068.stderr:----------------------------------------------------- 2026-02-26T03:26:16.368 INFO:teuthology.orchestra.run.trial068.stderr:Suppressions used: 2026-02-26T03:26:16.368 INFO:teuthology.orchestra.run.trial068.stderr: count bytes template 2026-02-26T03:26:16.368 INFO:teuthology.orchestra.run.trial068.stderr: 1 8 MallocExtension::Register 2026-02-26T03:26:16.368 INFO:teuthology.orchestra.run.trial068.stderr:----------------------------------------------------- 2026-02-26T03:26:16.368 INFO:teuthology.orchestra.run.trial068.stderr: 2026-02-26T03:26:16.387 INFO:teuthology.orchestra.run.trial068.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-26T03:26:16.387 INFO:teuthology.orchestra.run.trial068.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-26T03:26:16.435 INFO:teuthology.orchestra.run.trial068.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-26T03:26:16.435 INFO:teuthology.orchestra.run.trial068.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-26T03:26:16.442 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:16,439 seastar - Reactor backend: linux-aio 2026-02-26T03:26:16.453 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:16,450 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:16.453 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:16,450 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:16.457 INFO:teuthology.orchestra.run.trial068.stderr:==36499==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:26:16.458 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:16,454 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-26T03:26:16.514 INFO:teuthology.orchestra.run.trial068.stderr:DEBUG 2026-02-26 03:26:16,511 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-26T03:26:16.597 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0xb6c72f 0xc084af 0x7c931c1 0x7c93360 0x7bdaa69 0x7be98c4 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.597 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15e37 0x9c50e1a 0x9c511cc 0x9c51482 0x9c51a44 0x9c59eeb 0x9c5a5cb 0x9d43118 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.598 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119109 /lib64/libasan.so.8+0xc5f4 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9b2559b 0x9d0c53e 0x9d0c66c 0x9d3ea77 0x9d3f21a 0x9d496e7 0x78947dd 0x7be9c23 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.638 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc258 /lib64/libasan.so.8+0xc553 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe1d59b 0xe1d8f2 0x9d497c5 0x78947dd 0x7bea228 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa14d6 0x755a4ec 0x755a640 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.695 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f7a738 0x9bf83c1 0x9bf8bfc 0xecf14d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46f44 2026-02-26T03:26:16.735 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x2a558b9 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.735 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.735 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x2a5590c 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.855 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb051b /lib64/libasan.so.8+0xb1437 0xb69757 0x1090878 0xf2e0dc 0x9c55b3b 0x9d42c55 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.855 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x9c5f70a 0x9d25a33 0x9d25bfb 0x9d25d26 0x9d26250 0x9d42adb 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.855 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b377 /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xb69757 0x1090878 0xf2e0dc 0x9c50c5c 0x9c50f59 0x9c511cc 0x9c52ef3 0x9d368ca 0x9d3851c 0x9d38b76 0x9d390bc 0x9d4334a 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.938 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f7a738 0x9bf83c1 0x9bf8bfc 0xecf14d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46f44 2026-02-26T03:26:16.952 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129749 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.952 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151b5 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:16.952 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc657f /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:19.120 INFO:teuthology.orchestra.run.trial068.stderr:----------------------------------------------------- 2026-02-26T03:26:19.121 INFO:teuthology.orchestra.run.trial068.stderr:Suppressions used: 2026-02-26T03:26:19.121 INFO:teuthology.orchestra.run.trial068.stderr: count bytes template 2026-02-26T03:26:19.121 INFO:teuthology.orchestra.run.trial068.stderr: 1 8 MallocExtension::Register 2026-02-26T03:26:19.121 INFO:teuthology.orchestra.run.trial068.stderr:----------------------------------------------------- 2026-02-26T03:26:19.121 INFO:teuthology.orchestra.run.trial068.stderr: 2026-02-26T03:26:19.154 DEBUG:teuthology.orchestra.run.trial068:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-26T03:26:19.179 DEBUG:teuthology.orchestra.run.trial068:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:26:19.328 INFO:teuthology.orchestra.run.trial068.stderr:INFO  osd - OSD::main: parsing early config 2026-02-26T03:26:19.384 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:19,381 seastar - Reactor backend: linux-aio 2026-02-26T03:26:19.399 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:19,396 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:19.399 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:19,396 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:19.401 INFO:teuthology.orchestra.run.trial068.stderr:==36569==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:26:19.456 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:19,453 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-26T03:26:19.457 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:19,453 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-26T03:26:19.457 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:19,453 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-26T03:26:19.457 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:19,453 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-26T03:26:19.457 INFO:teuthology.orchestra.run.trial068.stderr:DEBUG 2026-02-26 03:26:19,453 [shard 0:main] seastar - reactor::drain 2026-02-26T03:26:21.507 INFO:teuthology.orchestra.run.trial068.stderr:----------------------------------------------------- 2026-02-26T03:26:21.507 INFO:teuthology.orchestra.run.trial068.stderr:Suppressions used: 2026-02-26T03:26:21.507 INFO:teuthology.orchestra.run.trial068.stderr: count bytes template 2026-02-26T03:26:21.507 INFO:teuthology.orchestra.run.trial068.stderr: 1 8 MallocExtension::Register 2026-02-26T03:26:21.507 INFO:teuthology.orchestra.run.trial068.stderr:----------------------------------------------------- 2026-02-26T03:26:21.507 INFO:teuthology.orchestra.run.trial068.stderr: 2026-02-26T03:26:21.526 INFO:teuthology.orchestra.run.trial068.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-26T03:26:21.527 INFO:teuthology.orchestra.run.trial068.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-26T03:26:21.569 INFO:teuthology.orchestra.run.trial068.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-26T03:26:21.569 INFO:teuthology.orchestra.run.trial068.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-26T03:26:21.576 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:21,573 seastar - Reactor backend: linux-aio 2026-02-26T03:26:21.586 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:21,583 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:21.586 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:21,583 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:26:21.590 INFO:teuthology.orchestra.run.trial068.stderr:==36567==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:26:21.590 INFO:teuthology.orchestra.run.trial068.stderr:INFO 2026-02-26 03:26:21,587 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-26T03:26:21.648 INFO:teuthology.orchestra.run.trial068.stderr:DEBUG 2026-02-26 03:26:21,645 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-26T03:26:21.728 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129797 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x9d25ae1 0x9d25bfb 0x9d25d26 0x9d26250 0x9d42adb 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:21.728 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x1125428 0xf2e0d1 0x9c50a8f 0x9c50f59 0x9c511cc 0x9c52ef3 0x9d320be 0x9d32837 0x9d32d22 0x9d330b5 0x9d45d3c 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:21.728 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc3ef /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe0b7b5 0xe0b9eb 0x7bea0a6 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:21.826 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151b9 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f7a738 0x9bf83c1 0x9bf8bfc 0xecf14d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46f44 2026-02-26T03:26:21.864 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc25 /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:21.865 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x7d837d2 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:21.865 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x2a00bea 0x2a559e0 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:21.985 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x1125428 0xf2e0d1 0x9d33899 0x9d33e86 0x9d34209 0x9d44360 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:21.985 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x10908ef 0xf2e0dc 0x9c50a8f 0x9c50f59 0x9c511cc 0x9c52ef3 0x9c53440 0x9c5363f 0x9d38961 0x9d390bc 0x9d4334a 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:21.986 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0xf2e11e 0x9c50c5c 0x9c50f59 0x9c511cc 0x9c52ef3 0x9d38e4d 0x9d390bc 0x9d4334a 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:22.068 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f7a738 0x9bf83c1 0x9bf8bfc 0xecf14d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46f44 2026-02-26T03:26:22.082 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ebc /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x2a55a3f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:22.082 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x2a5599f 0x7d837b9 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:22.082 INFO:teuthology.orchestra.run.trial068.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x7d837bd 0x7d9fd5e 0x7e419e3 0x7dc12dd 0x7877bf3 0x78788e8 0x79adc2a 0x7836cfc 0x78b1d9e 0x78fa531 0x78fa7aa 0x799b982 0x799bae1 0x79add8d 0x7868874 0x798a5f1 0x798a680 0x798a724 0x73e442a 0x7404ce9 0x740998a 0x754d7e2 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:26:24.226 INFO:teuthology.orchestra.run.trial068.stderr:----------------------------------------------------- 2026-02-26T03:26:24.227 INFO:teuthology.orchestra.run.trial068.stderr:Suppressions used: 2026-02-26T03:26:24.227 INFO:teuthology.orchestra.run.trial068.stderr: count bytes template 2026-02-26T03:26:24.227 INFO:teuthology.orchestra.run.trial068.stderr: 1 8 MallocExtension::Register 2026-02-26T03:26:24.227 INFO:teuthology.orchestra.run.trial068.stderr:----------------------------------------------------- 2026-02-26T03:26:24.227 INFO:teuthology.orchestra.run.trial068.stderr: 2026-02-26T03:26:24.260 DEBUG:teuthology.orchestra.run.trial068:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-26T03:26:24.285 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-26T03:26:24.285 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:26:24.285 DEBUG:teuthology.orchestra.run.trial012:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-26T03:26:24.311 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:26:24.311 DEBUG:teuthology.orchestra.run.trial012:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-26T03:26:24.376 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:26:24.376 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-26T03:26:24.403 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:26:24.403 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-26T03:26:24.467 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:26:24.467 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-26T03:26:24.531 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:26:24.532 DEBUG:teuthology.orchestra.run.trial012:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-26T03:26:24.550 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:26:24.550 DEBUG:teuthology.orchestra.run.trial068:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2026-02-26T03:26:24.587 INFO:tasks.ceph:Adding keys to all mons... 2026-02-26T03:26:24.588 DEBUG:teuthology.orchestra.run.trial012:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-26T03:26:24.612 INFO:teuthology.orchestra.run.trial012.stdout:[osd.0] 2026-02-26T03:26:24.613 INFO:teuthology.orchestra.run.trial012.stdout: key = AQBMvZ9pQAg8JhAA2/+J8k7ef7oSRCY0eN9ozQ== 2026-02-26T03:26:24.613 INFO:teuthology.orchestra.run.trial012.stdout:[osd.1] 2026-02-26T03:26:24.613 INFO:teuthology.orchestra.run.trial012.stdout: key = AQBRvZ9p/q5jKxAA2JzYSPpKHmGcF73OnHoCDw== 2026-02-26T03:26:24.613 INFO:teuthology.orchestra.run.trial012.stdout:[mgr.x] 2026-02-26T03:26:24.613 INFO:teuthology.orchestra.run.trial012.stdout: key = AQBIvZ9pITToGBAAHwS3z012XV4McKptXINJ2w== 2026-02-26T03:26:24.613 INFO:teuthology.orchestra.run.trial012.stdout:[osd.2] 2026-02-26T03:26:24.613 INFO:teuthology.orchestra.run.trial012.stdout: key = AQBYvZ9px8K4HhAAmoFfP+SenWzKIl3jd2O1og== 2026-02-26T03:26:24.613 INFO:teuthology.orchestra.run.trial012.stdout:[osd.3] 2026-02-26T03:26:24.613 INFO:teuthology.orchestra.run.trial012.stdout: key = AQBdvZ9px4mzJhAAg00UXO7SkDQBmjiJ97jO9Q== 2026-02-26T03:26:24.613 INFO:teuthology.orchestra.run.trial012.stdout:[client.0] 2026-02-26T03:26:24.614 INFO:teuthology.orchestra.run.trial012.stdout: key = AQBIvZ9p0FvAGxAACtOjukAoKvU7xV9SWvvFYQ== 2026-02-26T03:26:24.614 INFO:teuthology.orchestra.run.trial012.stdout:[client.1] 2026-02-26T03:26:24.614 INFO:teuthology.orchestra.run.trial012.stdout: key = AQBIvZ9pGvrGHhAAFcUMO3HxJOmDVieWDlGpNQ== 2026-02-26T03:26:24.615 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-26T03:26:24.703 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-26T03:26:24.752 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-26T03:26:24.800 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-26T03:26:24.848 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-26T03:26:24.897 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-26T03:26:24.948 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-26T03:26:24.999 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-26T03:26:24.999 DEBUG:teuthology.orchestra.run.trial012:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-26T03:26:25.024 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-26T03:26:25.111 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:25.109+0000 7f973c02ec00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.111 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:25.109+0000 7f973c02ec00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.123 DEBUG:teuthology.orchestra.run.trial012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-26T03:26:25.148 DEBUG:teuthology.orchestra.run.trial012:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:26:25.205 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-26T03:26:25.206 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-26T03:26:25.206 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-26T03:26:25.248 INFO:tasks.ceph.mon.a:Started 2026-02-26T03:26:25.248 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-26T03:26:25.248 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-26T03:26:25.249 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-26T03:26:25.252 INFO:tasks.ceph.mgr.x:Started 2026-02-26T03:26:25.252 DEBUG:tasks.ceph:set 0 configs 2026-02-26T03:26:25.252 DEBUG:teuthology.orchestra.run.trial012:> sudo ceph --cluster ceph config dump 2026-02-26T03:26:25.306 INFO:tasks.ceph.mon.a.trial012.stderr:2026-02-26T03:26:25.304+0000 7f5c06a36c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.306 INFO:tasks.ceph.mon.a.trial012.stderr:2026-02-26T03:26:25.304+0000 7f5c06a36c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.306 INFO:tasks.ceph.mon.a.trial012.stderr:2026-02-26T03:26:25.304+0000 7f5c06a36c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.310 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-26T03:26:25.307+0000 7f02567befc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.325 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:25.321+0000 7f30aa77e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.328 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:25.327+0000 7f30aa77e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.342 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-26T03:26:25.338+0000 7f02567befc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.342 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-26T03:26:25.339+0000 7f02567befc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.361 INFO:teuthology.orchestra.run.trial012.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-26T03:26:25.367 INFO:tasks.ceph:Setting crush tunables to default 2026-02-26T03:26:25.368 DEBUG:teuthology.orchestra.run.trial012:> sudo ceph --cluster ceph osd crush tunables default 2026-02-26T03:26:25.441 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:25.436+0000 7f680c8cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.444 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:25.442+0000 7f680c8cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.479 INFO:teuthology.orchestra.run.trial012.stderr:adjusted tunables profile to default 2026-02-26T03:26:25.487 INFO:tasks.ceph:check_enable_crimson: True 2026-02-26T03:26:25.487 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-26T03:26:25.487 DEBUG:teuthology.orchestra.run.trial012:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-26T03:26:25.562 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:25.557+0000 7efc5c1d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.564 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:25.563+0000 7efc5c1d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:25.602 INFO:tasks.ceph.mon.a.trial012.stderr:2026-02-26T03:26:25.601+0000 7f5bfddf1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-26T03:26:25.603 INFO:tasks.ceph.mon.a.trial012.stderr:Please be aware that this feature is experimental, untested, 2026-02-26T03:26:25.603 INFO:tasks.ceph.mon.a.trial012.stderr:unsupported, and may result in data corruption, data loss, 2026-02-26T03:26:25.603 INFO:tasks.ceph.mon.a.trial012.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-26T03:26:25.603 INFO:tasks.ceph.mon.a.trial012.stderr:feature with important data. 2026-02-26T03:26:25.603 INFO:tasks.ceph.mon.a.trial012.stderr: 2026-02-26T03:26:26.181 INFO:tasks.ceph.mgr.x.trial068.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-26T03:26:26.181 INFO:tasks.ceph.mgr.x.trial068.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-26T03:26:26.182 INFO:tasks.ceph.mgr.x.trial068.stderr: from numpy import show_config as show_numpy_config 2026-02-26T03:26:26.488 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-26T03:26:26.489 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:26:26.489 DEBUG:teuthology.orchestra.run.trial012:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-26T03:26:26.514 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:26:26.514 DEBUG:teuthology.orchestra.run.trial012:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-26T03:26:26.579 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:26:26.579 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-26T03:26:26.604 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-26T03:26:26.605 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-26T03:26:26.669 DEBUG:teuthology.orchestra.run.trial068:> sudo ceph --cluster ceph osd new 5fb2e3d6-0416-4aa3-ac72-b5bc69d2ac65 0 2026-02-26T03:26:26.782 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-26T03:26:26.776+0000 7f7c805ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:26.784 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-26T03:26:26.781+0000 7f7c805ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:26.822 INFO:teuthology.orchestra.run.trial068.stdout:0 2026-02-26T03:26:26.830 DEBUG:teuthology.orchestra.run.trial068:> sudo ceph --cluster ceph osd new 91f65545-a261-4939-87a5-33cf8e3fa19f 1 2026-02-26T03:26:26.902 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-26T03:26:26.895+0000 7f8180f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:26.904 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-26T03:26:26.901+0000 7f8180f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:26.942 INFO:teuthology.orchestra.run.trial068.stdout:1 2026-02-26T03:26:26.950 DEBUG:teuthology.orchestra.run.trial068:> sudo ceph --cluster ceph osd new e932d28c-883b-4fd0-a0a8-57d9a0f55eb7 2 2026-02-26T03:26:27.022 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-26T03:26:27.015+0000 7f91bbbb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:27.024 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-26T03:26:27.021+0000 7f91bbbb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:27.062 INFO:teuthology.orchestra.run.trial068.stdout:2 2026-02-26T03:26:27.070 DEBUG:teuthology.orchestra.run.trial068:> sudo ceph --cluster ceph osd new 7adc0e07-7937-4690-bb26-ec324ee9dad9 3 2026-02-26T03:26:27.146 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-26T03:26:27.139+0000 7f0d4c4c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:27.148 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-26T03:26:27.145+0000 7f0d4c4c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:27.185 INFO:teuthology.orchestra.run.trial068.stdout:3 2026-02-26T03:26:27.193 DEBUG:teuthology.orchestra.run.trial012:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-26T03:26:27.222 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-26T03:26:27.222 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-26T03:26:27.264 INFO:tasks.ceph.osd.0:Started 2026-02-26T03:26:27.264 DEBUG:teuthology.orchestra.run.trial012:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-26T03:26:27.292 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-26T03:26:27.292 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-26T03:26:27.333 INFO:tasks.ceph.osd.1:Started 2026-02-26T03:26:27.334 DEBUG:teuthology.orchestra.run.trial068:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-26T03:26:27.364 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-26T03:26:27.364 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-26T03:26:27.406 INFO:tasks.ceph.osd.2:Started 2026-02-26T03:26:27.406 DEBUG:teuthology.orchestra.run.trial068:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2026-02-26T03:26:27.434 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-26T03:26:27.434 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2026-02-26T03:26:27.477 INFO:tasks.ceph.osd.3:Started 2026-02-26T03:26:27.477 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-26T03:26:27.552 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:27.548+0000 7f96bb05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:27.554 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:27.553+0000 7f96bb05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:27.587 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:26:27.588 INFO:teuthology.orchestra.run.trial012.stdout:{"epoch":7,"fsid":"dc070458-d616-4479-8618-4ead68863657","created":"2026-02-26T03:26:25.317244+0000","modified":"2026-02-26T03:26:27.184265+0000","last_up_change":"0.000000","last_in_change":"2026-02-26T03:26:27.184265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"5fb2e3d6-0416-4aa3-ac72-b5bc69d2ac65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"91f65545-a261-4939-87a5-33cf8e3fa19f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e932d28c-883b-4fd0-a0a8-57d9a0f55eb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7adc0e07-7937-4690-bb26-ec324ee9dad9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:26:27.596 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-26T03:26:27.596 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-26T03:26:28.000 DEBUG:teuthology.orchestra.run.trial012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-26T03:26:28.066 INFO:teuthology.misc.health.trial012.stderr:2026-02-26T03:26:28.062+0000 7f30f081a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:28.069 INFO:teuthology.misc.health.trial012.stderr:2026-02-26T03:26:28.067+0000 7f30f081a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:28.102 INFO:teuthology.misc.health.trial012.stdout: 2026-02-26T03:26:28.103 INFO:teuthology.misc.health.trial012.stdout:{"epoch":7,"fsid":"dc070458-d616-4479-8618-4ead68863657","created":"2026-02-26T03:26:25.317244+0000","modified":"2026-02-26T03:26:27.184265+0000","last_up_change":"0.000000","last_in_change":"2026-02-26T03:26:27.184265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"5fb2e3d6-0416-4aa3-ac72-b5bc69d2ac65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"91f65545-a261-4939-87a5-33cf8e3fa19f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e932d28c-883b-4fd0-a0a8-57d9a0f55eb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7adc0e07-7937-4690-bb26-ec324ee9dad9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:26:28.109 DEBUG:teuthology.misc:0 of 4 OSDs are up 2026-02-26T03:26:28.551 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-26T03:26:28.547+0000 7f0171697640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-26T03:26:34.513 DEBUG:teuthology.orchestra.run.trial012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-26T03:26:34.583 INFO:teuthology.misc.health.trial012.stderr:2026-02-26T03:26:34.579+0000 7f874185a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:34.586 INFO:teuthology.misc.health.trial012.stderr:2026-02-26T03:26:34.584+0000 7f874185a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:34.698 INFO:teuthology.misc.health.trial012.stdout: 2026-02-26T03:26:34.698 INFO:teuthology.misc.health.trial012.stdout:{"epoch":11,"fsid":"dc070458-d616-4479-8618-4ead68863657","created":"2026-02-26T03:26:25.317244+0000","modified":"2026-02-26T03:26:34.618095+0000","last_up_change":"2026-02-26T03:26:34.618095+0000","last_in_change":"2026-02-26T03:26:27.184265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"5fb2e3d6-0416-4aa3-ac72-b5bc69d2ac65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6804","nonce":3513533170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6803","nonce":3513533170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6807","nonce":3513533170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6800","nonce":3513533170}]},"public_addr":"10.20.193.12:6804/3513533170","cluster_addr":"10.20.193.12:6803/3513533170","heartbeat_back_addr":"10.20.193.12:6807/3513533170","heartbeat_front_addr":"10.20.193.12:6800/3513533170","state":["exists","up"]},{"osd":1,"uuid":"91f65545-a261-4939-87a5-33cf8e3fa19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6801","nonce":2473255298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6802","nonce":2473255298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6806","nonce":2473255298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6805","nonce":2473255298}]},"public_addr":"10.20.193.12:6801/2473255298","cluster_addr":"10.20.193.12:6802/2473255298","heartbeat_back_addr":"10.20.193.12:6806/2473255298","heartbeat_front_addr":"10.20.193.12:6805/2473255298","state":["exists","up"]},{"osd":2,"uuid":"e932d28c-883b-4fd0-a0a8-57d9a0f55eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6804","nonce":256615363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6803","nonce":256615363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6802","nonce":256615363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6805","nonce":256615363}]},"public_addr":"10.20.193.68:6804/256615363","cluster_addr":"10.20.193.68:6803/256615363","heartbeat_back_addr":"10.20.193.68:6802/256615363","heartbeat_front_addr":"10.20.193.68:6805/256615363","state":["exists","up"]},{"osd":3,"uuid":"7adc0e07-7937-4690-bb26-ec324ee9dad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6810","nonce":1576658665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6809","nonce":1576658665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6808","nonce":1576658665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6807","nonce":1576658665}]},"public_addr":"10.20.193.68:6810/1576658665","cluster_addr":"10.20.193.68:6809/1576658665","heartbeat_back_addr":"10.20.193.68:6808/1576658665","heartbeat_front_addr":"10.20.193.68:6807/1576658665","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:26:34.707 DEBUG:teuthology.misc:4 of 4 OSDs are up 2026-02-26T03:26:34.708 INFO:tasks.ceph:Creating RBD pool 2026-02-26T03:26:34.708 DEBUG:teuthology.orchestra.run.trial012:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-26T03:26:34.782 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:34.777+0000 7fbe3a83d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:34.784 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:34.783+0000 7fbe3a83d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:35.626 INFO:teuthology.orchestra.run.trial012.stderr:pool 'rbd' created 2026-02-26T03:26:35.637 DEBUG:teuthology.orchestra.run.trial012:> rbd --cluster ceph pool init rbd 2026-02-26T03:26:35.680 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:35.677+0000 7fed3f891240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:35.680 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:35.677+0000 7fed3f891240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:35.683 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:35.681+0000 7fed3f891240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:38.666 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-26T03:26:38.667 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-26T03:26:38.667 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-26T03:26:38.781 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:38.776+0000 7f9738a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:38.784 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:38.781+0000 7f9738a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:38.895 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:26:38.904 INFO:teuthology.orchestra.run.trial012.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-26T03:26:38.904 INFO:tasks.ceph_manager:config epoch is 1 2026-02-26T03:26:38.904 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-26T03:26:38.904 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-26T03:26:38.905 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-26T03:26:38.980 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:38.975+0000 7fbf894c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:38.983 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:38.980+0000 7fbf894c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.124 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:26:39.133 INFO:teuthology.orchestra.run.trial012.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6800","nonce":1519597090},{"type":"v1","addr":"10.20.193.68:6801","nonce":1519597090}]},"active_addr":"10.20.193.68:6801/1519597090","active_change":"2026-02-26T03:26:27.544350+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.68:0","nonce":2317954589}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.68:0","nonce":660362541}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.68:0","nonce":2294971175}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.68:0","nonce":3608198006}]}]} 2026-02-26T03:26:39.141 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-26T03:26:39.141 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-26T03:26:39.141 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-26T03:26:39.217 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.213+0000 7f3c74174640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.220 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.217+0000 7f3c74174640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.332 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:26:39.333 INFO:teuthology.orchestra.run.trial012.stdout:{"epoch":15,"fsid":"dc070458-d616-4479-8618-4ead68863657","created":"2026-02-26T03:26:25.317244+0000","modified":"2026-02-26T03:26:38.634071+0000","last_up_change":"2026-02-26T03:26:34.618095+0000","last_in_change":"2026-02-26T03:26:27.184265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-26T03:26:34.898551+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2026-02-26T03:26:35.564040+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fb2e3d6-0416-4aa3-ac72-b5bc69d2ac65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6804","nonce":3513533170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6803","nonce":3513533170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6807","nonce":3513533170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6800","nonce":3513533170}]},"public_addr":"10.20.193.12:6804/3513533170","cluster_addr":"10.20.193.12:6803/3513533170","heartbeat_back_addr":"10.20.193.12:6807/3513533170","heartbeat_front_addr":"10.20.193.12:6800/3513533170","state":["exists","up"]},{"osd":1,"uuid":"91f65545-a261-4939-87a5-33cf8e3fa19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6801","nonce":2473255298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6802","nonce":2473255298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6806","nonce":2473255298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6805","nonce":2473255298}]},"public_addr":"10.20.193.12:6801/2473255298","cluster_addr":"10.20.193.12:6802/2473255298","heartbeat_back_addr":"10.20.193.12:6806/2473255298","heartbeat_front_addr":"10.20.193.12:6805/2473255298","state":["exists","up"]},{"osd":2,"uuid":"e932d28c-883b-4fd0-a0a8-57d9a0f55eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6804","nonce":256615363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6803","nonce":256615363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6802","nonce":256615363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6805","nonce":256615363}]},"public_addr":"10.20.193.68:6804/256615363","cluster_addr":"10.20.193.68:6803/256615363","heartbeat_back_addr":"10.20.193.68:6802/256615363","heartbeat_front_addr":"10.20.193.68:6805/256615363","state":["exists","up"]},{"osd":3,"uuid":"7adc0e07-7937-4690-bb26-ec324ee9dad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6810","nonce":1576658665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6809","nonce":1576658665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6808","nonce":1576658665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6807","nonce":1576658665}]},"public_addr":"10.20.193.68:6810/1576658665","cluster_addr":"10.20.193.68:6809/1576658665","heartbeat_back_addr":"10.20.193.68:6808/1576658665","heartbeat_front_addr":"10.20.193.68:6807/1576658665","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:26:39.342 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-26T03:26:39.343 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-26T03:26:39.417 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.412+0000 7f9a72a3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.419 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.417+0000 7f9a72a3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.532 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:26:39.532 INFO:teuthology.orchestra.run.trial012.stdout:{"epoch":15,"fsid":"dc070458-d616-4479-8618-4ead68863657","created":"2026-02-26T03:26:25.317244+0000","modified":"2026-02-26T03:26:38.634071+0000","last_up_change":"2026-02-26T03:26:34.618095+0000","last_in_change":"2026-02-26T03:26:27.184265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-26T03:26:34.898551+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2026-02-26T03:26:35.564040+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fb2e3d6-0416-4aa3-ac72-b5bc69d2ac65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6804","nonce":3513533170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6803","nonce":3513533170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6807","nonce":3513533170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6800","nonce":3513533170}]},"public_addr":"10.20.193.12:6804/3513533170","cluster_addr":"10.20.193.12:6803/3513533170","heartbeat_back_addr":"10.20.193.12:6807/3513533170","heartbeat_front_addr":"10.20.193.12:6800/3513533170","state":["exists","up"]},{"osd":1,"uuid":"91f65545-a261-4939-87a5-33cf8e3fa19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6801","nonce":2473255298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6802","nonce":2473255298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6806","nonce":2473255298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6805","nonce":2473255298}]},"public_addr":"10.20.193.12:6801/2473255298","cluster_addr":"10.20.193.12:6802/2473255298","heartbeat_back_addr":"10.20.193.12:6806/2473255298","heartbeat_front_addr":"10.20.193.12:6805/2473255298","state":["exists","up"]},{"osd":2,"uuid":"e932d28c-883b-4fd0-a0a8-57d9a0f55eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6804","nonce":256615363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6803","nonce":256615363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6802","nonce":256615363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6805","nonce":256615363}]},"public_addr":"10.20.193.68:6804/256615363","cluster_addr":"10.20.193.68:6803/256615363","heartbeat_back_addr":"10.20.193.68:6802/256615363","heartbeat_front_addr":"10.20.193.68:6805/256615363","state":["exists","up"]},{"osd":3,"uuid":"7adc0e07-7937-4690-bb26-ec324ee9dad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6810","nonce":1576658665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6809","nonce":1576658665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6808","nonce":1576658665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6807","nonce":1576658665}]},"public_addr":"10.20.193.68:6810/1576658665","cluster_addr":"10.20.193.68:6809/1576658665","heartbeat_back_addr":"10.20.193.68:6808/1576658665","heartbeat_front_addr":"10.20.193.68:6807/1576658665","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:26:39.542 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-26T03:26:39.543 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-26T03:26:39.543 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-26T03:26:39.544 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-26T03:26:39.626 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.621+0000 7f9c7230b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.627 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.621+0000 7f3595811640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.627 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.621+0000 7f845119c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.628 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.622+0000 7fa1b22cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.629 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.626+0000 7f845119c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.629 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.626+0000 7f3595811640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.629 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.627+0000 7f9c7230b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.630 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.627+0000 7fa1b22cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.647 INFO:teuthology.orchestra.run.trial012.stdout:47244640261 2026-02-26T03:26:39.647 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-26T03:26:39.648 INFO:teuthology.orchestra.run.trial012.stdout:47244640261 2026-02-26T03:26:39.649 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-26T03:26:39.650 INFO:teuthology.orchestra.run.trial012.stdout:42949672966 2026-02-26T03:26:39.650 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-26T03:26:39.651 INFO:teuthology.orchestra.run.trial012.stdout:47244640261 2026-02-26T03:26:39.652 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-26T03:26:39.771 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.766+0000 7fcd81d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.771 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.766+0000 7f84f768b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.772 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.765+0000 7f631fdb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.772 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.766+0000 7ff5f261a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.774 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.771+0000 7fcd81d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.774 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.771+0000 7f84f768b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.774 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.771+0000 7f631fdb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.775 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:39.772+0000 7ff5f261a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:39.887 INFO:teuthology.orchestra.run.trial012.stdout:47244640261 2026-02-26T03:26:39.888 INFO:teuthology.orchestra.run.trial012.stdout:47244640261 2026-02-26T03:26:39.892 INFO:teuthology.orchestra.run.trial012.stdout:47244640261 2026-02-26T03:26:39.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.3 2026-02-26T03:26:39.898 DEBUG:teuthology.parallel:result is None 2026-02-26T03:26:39.900 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.2 2026-02-26T03:26:39.901 DEBUG:teuthology.parallel:result is None 2026-02-26T03:26:39.901 INFO:teuthology.orchestra.run.trial012.stdout:42949672966 2026-02-26T03:26:39.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.0 2026-02-26T03:26:39.903 DEBUG:teuthology.parallel:result is None 2026-02-26T03:26:39.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.1 2026-02-26T03:26:39.912 DEBUG:teuthology.parallel:result is None 2026-02-26T03:26:39.912 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-26T03:26:39.912 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:26:40.027 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:40.022+0000 7fac2101a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:40.029 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:40.027+0000 7fac2101a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:40.141 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:26:40.141 INFO:teuthology.orchestra.run.trial012.stderr:dumped all 2026-02-26T03:26:40.150 INFO:teuthology.orchestra.run.trial012.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2026-02-26T03:26:39.637789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":2720,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208663904,"statfs":{"total":213674622976,"available":213671837696,"internally_reserved":0,"allocated":2785280,"data_stored":2785280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.643674+0000","last_change":"2026-02-26T03:26:38.647488+0000","last_active":"2026-02-26T03:26:38.643674+0000","last_peered":"2026-02-26T03:26:38.643674+0000","last_clean":"2026-02-26T03:26:38.643674+0000","last_became_active":"2026-02-26T03:26:36.663643+0000","last_became_peered":"2026-02-26T03:26:36.663643+0000","last_unstale":"2026-02-26T03:26:38.643674+0000","last_undegraded":"2026-02-26T03:26:38.643674+0000","last_fullsized":"2026-02-26T03:26:38.643674+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.645690+0000","last_change":"2026-02-26T03:26:38.647886+0000","last_active":"2026-02-26T03:26:38.645690+0000","last_peered":"2026-02-26T03:26:38.645690+0000","last_clean":"2026-02-26T03:26:38.645690+0000","last_became_active":"2026-02-26T03:26:36.667431+0000","last_became_peered":"2026-02-26T03:26:36.667431+0000","last_unstale":"2026-02-26T03:26:38.645690+0000","last_undegraded":"2026-02-26T03:26:38.645690+0000","last_fullsized":"2026-02-26T03:26:38.645690+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.645496+0000","last_change":"2026-02-26T03:26:38.647940+0000","last_active":"2026-02-26T03:26:38.645496+0000","last_peered":"2026-02-26T03:26:38.645496+0000","last_clean":"2026-02-26T03:26:38.645496+0000","last_became_active":"2026-02-26T03:26:36.672454+0000","last_became_peered":"2026-02-26T03:26:36.672454+0000","last_unstale":"2026-02-26T03:26:38.645496+0000","last_undegraded":"2026-02-26T03:26:38.645496+0000","last_fullsized":"2026-02-26T03:26:38.645496+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.645056+0000","last_change":"2026-02-26T03:26:38.646875+0000","last_active":"2026-02-26T03:26:38.645056+0000","last_peered":"2026-02-26T03:26:38.645056+0000","last_clean":"2026-02-26T03:26:38.645056+0000","last_became_active":"2026-02-26T03:26:36.662567+0000","last_became_peered":"2026-02-26T03:26:36.662567+0000","last_unstale":"2026-02-26T03:26:38.645056+0000","last_undegraded":"2026-02-26T03:26:38.645056+0000","last_fullsized":"2026-02-26T03:26:38.645056+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.646822+0000","last_change":"2026-02-26T03:26:36.680586+0000","last_active":"2026-02-26T03:26:38.646822+0000","last_peered":"2026-02-26T03:26:38.646822+0000","last_clean":"2026-02-26T03:26:38.646822+0000","last_became_active":"2026-02-26T03:26:36.667237+0000","last_became_peered":"2026-02-26T03:26:36.667237+0000","last_unstale":"2026-02-26T03:26:38.646822+0000","last_undegraded":"2026-02-26T03:26:38.646822+0000","last_fullsized":"2026-02-26T03:26:38.646822+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.642803+0000","last_change":"2026-02-26T03:26:38.645479+0000","last_active":"2026-02-26T03:26:38.642803+0000","last_peered":"2026-02-26T03:26:38.642803+0000","last_clean":"2026-02-26T03:26:38.642803+0000","last_became_active":"2026-02-26T03:26:36.666405+0000","last_became_peered":"2026-02-26T03:26:36.666405+0000","last_unstale":"2026-02-26T03:26:38.642803+0000","last_undegraded":"2026-02-26T03:26:38.642803+0000","last_fullsized":"2026-02-26T03:26:38.642803+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.646292+0000","last_change":"2026-02-26T03:26:38.646292+0000","last_active":"2026-02-26T03:26:38.646292+0000","last_peered":"2026-02-26T03:26:38.646292+0000","last_clean":"2026-02-26T03:26:38.646292+0000","last_became_active":"2026-02-26T03:26:36.660207+0000","last_became_peered":"2026-02-26T03:26:36.660207+0000","last_unstale":"2026-02-26T03:26:38.646292+0000","last_undegraded":"2026-02-26T03:26:38.646292+0000","last_fullsized":"2026-02-26T03:26:38.646292+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.645918+0000","last_change":"2026-02-26T03:26:38.648362+0000","last_active":"2026-02-26T03:26:38.645918+0000","last_peered":"2026-02-26T03:26:38.645918+0000","last_clean":"2026-02-26T03:26:38.645918+0000","last_became_active":"2026-02-26T03:26:36.669960+0000","last_became_peered":"2026-02-26T03:26:36.669960+0000","last_unstale":"2026-02-26T03:26:38.645918+0000","last_undegraded":"2026-02-26T03:26:38.645918+0000","last_fullsized":"2026-02-26T03:26:38.645918+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.643158+0000","last_change":"2026-02-26T03:26:38.647558+0000","last_active":"2026-02-26T03:26:38.643158+0000","last_peered":"2026-02-26T03:26:38.643158+0000","last_clean":"2026-02-26T03:26:38.643158+0000","last_became_active":"2026-02-26T03:26:36.658652+0000","last_became_peered":"2026-02-26T03:26:36.658652+0000","last_unstale":"2026-02-26T03:26:38.643158+0000","last_undegraded":"2026-02-26T03:26:38.643158+0000","last_fullsized":"2026-02-26T03:26:38.643158+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417574400,"internally_reserved":0,"allocated":1081344,"data_stored":1081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253509632,"internally_reserved":0,"allocated":2457600,"data_stored":2457600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1056,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165600,"statfs":{"total":53418655744,"available":53417574400,"internally_reserved":0,"allocated":1081344,"data_stored":1081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":320,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166336,"statfs":{"total":53418655744,"available":53418328064,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1024,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165632,"statfs":{"total":53418655744,"available":53417607168,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":320,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166336,"statfs":{"total":53418655744,"available":53418328064,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417607168,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53418328064,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53417574400,"internally_reserved":0,"allocated":1081344,"data_stored":1081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53418655744,"available":53417574400,"internally_reserved":0,"allocated":1081344,"data_stored":1081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:26:40.151 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:26:40.227 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:40.222+0000 7f33e141a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:40.229 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:40.227+0000 7f33e141a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:40.340 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:26:40.341 INFO:teuthology.orchestra.run.trial012.stderr:dumped all 2026-02-26T03:26:40.350 INFO:teuthology.orchestra.run.trial012.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2026-02-26T03:26:39.637789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":2720,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208663904,"statfs":{"total":213674622976,"available":213671837696,"internally_reserved":0,"allocated":2785280,"data_stored":2785280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.643674+0000","last_change":"2026-02-26T03:26:38.647488+0000","last_active":"2026-02-26T03:26:38.643674+0000","last_peered":"2026-02-26T03:26:38.643674+0000","last_clean":"2026-02-26T03:26:38.643674+0000","last_became_active":"2026-02-26T03:26:36.663643+0000","last_became_peered":"2026-02-26T03:26:36.663643+0000","last_unstale":"2026-02-26T03:26:38.643674+0000","last_undegraded":"2026-02-26T03:26:38.643674+0000","last_fullsized":"2026-02-26T03:26:38.643674+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.645690+0000","last_change":"2026-02-26T03:26:38.647886+0000","last_active":"2026-02-26T03:26:38.645690+0000","last_peered":"2026-02-26T03:26:38.645690+0000","last_clean":"2026-02-26T03:26:38.645690+0000","last_became_active":"2026-02-26T03:26:36.667431+0000","last_became_peered":"2026-02-26T03:26:36.667431+0000","last_unstale":"2026-02-26T03:26:38.645690+0000","last_undegraded":"2026-02-26T03:26:38.645690+0000","last_fullsized":"2026-02-26T03:26:38.645690+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.645496+0000","last_change":"2026-02-26T03:26:38.647940+0000","last_active":"2026-02-26T03:26:38.645496+0000","last_peered":"2026-02-26T03:26:38.645496+0000","last_clean":"2026-02-26T03:26:38.645496+0000","last_became_active":"2026-02-26T03:26:36.672454+0000","last_became_peered":"2026-02-26T03:26:36.672454+0000","last_unstale":"2026-02-26T03:26:38.645496+0000","last_undegraded":"2026-02-26T03:26:38.645496+0000","last_fullsized":"2026-02-26T03:26:38.645496+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.645056+0000","last_change":"2026-02-26T03:26:38.646875+0000","last_active":"2026-02-26T03:26:38.645056+0000","last_peered":"2026-02-26T03:26:38.645056+0000","last_clean":"2026-02-26T03:26:38.645056+0000","last_became_active":"2026-02-26T03:26:36.662567+0000","last_became_peered":"2026-02-26T03:26:36.662567+0000","last_unstale":"2026-02-26T03:26:38.645056+0000","last_undegraded":"2026-02-26T03:26:38.645056+0000","last_fullsized":"2026-02-26T03:26:38.645056+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.646822+0000","last_change":"2026-02-26T03:26:36.680586+0000","last_active":"2026-02-26T03:26:38.646822+0000","last_peered":"2026-02-26T03:26:38.646822+0000","last_clean":"2026-02-26T03:26:38.646822+0000","last_became_active":"2026-02-26T03:26:36.667237+0000","last_became_peered":"2026-02-26T03:26:36.667237+0000","last_unstale":"2026-02-26T03:26:38.646822+0000","last_undegraded":"2026-02-26T03:26:38.646822+0000","last_fullsized":"2026-02-26T03:26:38.646822+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.642803+0000","last_change":"2026-02-26T03:26:38.645479+0000","last_active":"2026-02-26T03:26:38.642803+0000","last_peered":"2026-02-26T03:26:38.642803+0000","last_clean":"2026-02-26T03:26:38.642803+0000","last_became_active":"2026-02-26T03:26:36.666405+0000","last_became_peered":"2026-02-26T03:26:36.666405+0000","last_unstale":"2026-02-26T03:26:38.642803+0000","last_undegraded":"2026-02-26T03:26:38.642803+0000","last_fullsized":"2026-02-26T03:26:38.642803+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.646292+0000","last_change":"2026-02-26T03:26:38.646292+0000","last_active":"2026-02-26T03:26:38.646292+0000","last_peered":"2026-02-26T03:26:38.646292+0000","last_clean":"2026-02-26T03:26:38.646292+0000","last_became_active":"2026-02-26T03:26:36.660207+0000","last_became_peered":"2026-02-26T03:26:36.660207+0000","last_unstale":"2026-02-26T03:26:38.646292+0000","last_undegraded":"2026-02-26T03:26:38.646292+0000","last_fullsized":"2026-02-26T03:26:38.646292+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.645918+0000","last_change":"2026-02-26T03:26:38.648362+0000","last_active":"2026-02-26T03:26:38.645918+0000","last_peered":"2026-02-26T03:26:38.645918+0000","last_clean":"2026-02-26T03:26:38.645918+0000","last_became_active":"2026-02-26T03:26:36.669960+0000","last_became_peered":"2026-02-26T03:26:36.669960+0000","last_unstale":"2026-02-26T03:26:38.645918+0000","last_undegraded":"2026-02-26T03:26:38.645918+0000","last_fullsized":"2026-02-26T03:26:38.645918+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:26:38.643158+0000","last_change":"2026-02-26T03:26:38.647558+0000","last_active":"2026-02-26T03:26:38.643158+0000","last_peered":"2026-02-26T03:26:38.643158+0000","last_clean":"2026-02-26T03:26:38.643158+0000","last_became_active":"2026-02-26T03:26:36.658652+0000","last_became_peered":"2026-02-26T03:26:36.658652+0000","last_unstale":"2026-02-26T03:26:38.643158+0000","last_undegraded":"2026-02-26T03:26:38.643158+0000","last_fullsized":"2026-02-26T03:26:38.643158+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417574400,"internally_reserved":0,"allocated":1081344,"data_stored":1081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253509632,"internally_reserved":0,"allocated":2457600,"data_stored":2457600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1056,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165600,"statfs":{"total":53418655744,"available":53417574400,"internally_reserved":0,"allocated":1081344,"data_stored":1081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":320,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166336,"statfs":{"total":53418655744,"available":53418328064,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1024,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165632,"statfs":{"total":53418655744,"available":53417607168,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":320,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166336,"statfs":{"total":53418655744,"available":53418328064,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417607168,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53418328064,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53417574400,"internally_reserved":0,"allocated":1081344,"data_stored":1081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53418655744,"available":53417574400,"internally_reserved":0,"allocated":1081344,"data_stored":1081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:26:40.351 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-26T03:26:40.352 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-26T03:26:40.352 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-26T03:26:40.352 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-26T03:26:40.427 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:40.423+0000 7fd9f561a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:40.429 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:26:40.426+0000 7fd9f561a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:26:40.566 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:26:40.567 INFO:teuthology.orchestra.run.trial012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-26T03:26:40.576 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-26T03:26:40.576 INFO:teuthology.run_tasks:Running task workunit... 2026-02-26T03:26:40.601 INFO:tasks.workunit:Pulling workunits from ref 1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996 2026-02-26T03:26:40.602 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-26T03:26:40.602 DEBUG:teuthology.orchestra.run.trial012:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-26T03:26:40.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:26:40.619 INFO:teuthology.orchestra.run.trial012.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-26T03:26:40.620 DEBUG:teuthology.orchestra.run.trial012:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-26T03:26:40.676 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-26T03:26:40.677 DEBUG:teuthology.orchestra.run.trial012:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-26T03:26:40.733 INFO:tasks.workunit:timeout=1h 2026-02-26T03:26:40.733 INFO:tasks.workunit:cleanup=True 2026-02-26T03:26:40.734 DEBUG:teuthology.orchestra.run.trial012:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996 2026-02-26T03:26:40.796 INFO:tasks.workunit.client.0.trial012.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-26T03:27:03.956 INFO:tasks.workunit.client.0.trial012.stderr:Note: switching to '1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996'. 2026-02-26T03:27:03.957 INFO:tasks.workunit.client.0.trial012.stderr: 2026-02-26T03:27:03.957 INFO:tasks.workunit.client.0.trial012.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-26T03:27:03.957 INFO:tasks.workunit.client.0.trial012.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-26T03:27:03.957 INFO:tasks.workunit.client.0.trial012.stderr:state without impacting any branches by switching back to a branch. 2026-02-26T03:27:03.957 INFO:tasks.workunit.client.0.trial012.stderr: 2026-02-26T03:27:03.957 INFO:tasks.workunit.client.0.trial012.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-26T03:27:03.957 INFO:tasks.workunit.client.0.trial012.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-26T03:27:03.957 INFO:tasks.workunit.client.0.trial012.stderr: 2026-02-26T03:27:03.957 INFO:tasks.workunit.client.0.trial012.stderr: git switch -c 2026-02-26T03:27:03.957 INFO:tasks.workunit.client.0.trial012.stderr: 2026-02-26T03:27:03.958 INFO:tasks.workunit.client.0.trial012.stderr:Or undo this operation with: 2026-02-26T03:27:03.958 INFO:tasks.workunit.client.0.trial012.stderr: 2026-02-26T03:27:03.958 INFO:tasks.workunit.client.0.trial012.stderr: git switch - 2026-02-26T03:27:03.958 INFO:tasks.workunit.client.0.trial012.stderr: 2026-02-26T03:27:03.958 INFO:tasks.workunit.client.0.trial012.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-26T03:27:03.958 INFO:tasks.workunit.client.0.trial012.stderr: 2026-02-26T03:27:03.958 INFO:tasks.workunit.client.0.trial012.stderr:HEAD is now at 1f87b69b1a8 Merge pull request #67522 from rhcs-dashboard/notification-banner-simple-mode 2026-02-26T03:27:03.959 DEBUG:teuthology.orchestra.run.trial012:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-26T03:27:04.017 INFO:tasks.workunit.client.0.trial012.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-26T03:27:04.019 INFO:tasks.workunit.client.0.trial012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-26T03:27:04.019 INFO:tasks.workunit.client.0.trial012.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-26T03:27:04.117 INFO:tasks.workunit.client.0.trial012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-26T03:27:04.148 INFO:tasks.workunit.client.0.trial012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-26T03:27:04.173 INFO:tasks.workunit.client.0.trial012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-26T03:27:04.174 INFO:tasks.workunit.client.0.trial012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-26T03:27:04.174 INFO:tasks.workunit.client.0.trial012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-26T03:27:04.198 INFO:tasks.workunit.client.0.trial012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-26T03:27:04.202 DEBUG:teuthology.orchestra.run.trial012:> set -ex 2026-02-26T03:27:04.202 DEBUG:teuthology.orchestra.run.trial012:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-26T03:27:04.259 INFO:tasks.workunit:Running workunits matching rados/test_python.sh on client.0... 2026-02-26T03:27:04.262 INFO:tasks.workunit:Running workunit rados/test_python.sh... 2026-02-26T03:27:04.262 DEBUG:teuthology.orchestra.run.trial012:workunit test rados/test_python.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 1h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh -m 'not (wait or tier or ec)' 2026-02-26T03:27:04.319 INFO:tasks.workunit.client.0.trial012.stderr:+ ceph osd pool create rbd 2026-02-26T03:27:04.373 INFO:tasks.workunit.client.0.trial012.stderr:2026-02-26T03:27:04.368+0000 7f8d2e5dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:27:04.375 INFO:tasks.workunit.client.0.trial012.stderr:2026-02-26T03:27:04.374+0000 7f8d2e5dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:27:04.542 INFO:tasks.workunit.client.0.trial012.stderr:pool 'rbd' already exists 2026-02-26T03:27:04.549 INFO:tasks.workunit.client.0.trial012.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh 2026-02-26T03:27:04.550 INFO:tasks.workunit.client.0.trial012.stderr:+ python3 -m pytest -v /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/../../../src/test/pybind/test_rados.py -m 'not (wait or tier or ec)' 2026-02-26T03:27:04.624 INFO:tasks.workunit.client.0.trial012.stdout:============================= test session starts ============================== 2026-02-26T03:27:04.624 INFO:tasks.workunit.client.0.trial012.stdout:platform linux -- Python 3.9.25, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3 2026-02-26T03:27:04.624 INFO:tasks.workunit.client.0.trial012.stdout:cachedir: .pytest_cache 2026-02-26T03:27:04.624 INFO:tasks.workunit.client.0.trial012.stdout:rootdir: /home/ubuntu/cephtest/clone.client.0/src/test/pybind, configfile: pytest.ini 2026-02-26T03:27:04.760 INFO:tasks.workunit.client.0.trial012.stdout:collecting ... collected 91 items / 5 deselected / 86 selected 2026-02-26T03:27:04.760 INFO:tasks.workunit.client.0.trial012.stdout: 2026-02-26T03:27:04.766 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_rados_init_error PASSED [ 1%] 2026-02-26T03:27:04.814 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_rados_init PASSED [ 2%] 2026-02-26T03:27:04.826 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_ioctx_context_manager PASSED [ 3%] 2026-02-26T03:27:04.833 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_parse_argv PASSED [ 4%] 2026-02-26T03:27:04.836 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_parse_argv_empty_str PASSED [ 5%] 2026-02-26T03:27:04.843 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_configuring PASSED [ 6%] 2026-02-26T03:27:04.856 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_connected PASSED [ 8%] 2026-02-26T03:27:04.867 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_shutdown PASSED [ 9%] 2026-02-26T03:27:04.883 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_ping_monitor PASSED [ 10%] 2026-02-26T03:27:04.898 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_annotations PASSED [ 11%] 2026-02-26T03:27:06.666 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_create PASSED [ 12%] 2026-02-26T03:27:08.673 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_create_utf8 PASSED [ 13%] 2026-02-26T03:27:10.681 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_pool_lookup_utf8 PASSED [ 15%] 2026-02-26T03:27:12.688 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_eexist PASSED [ 16%] 2026-02-26T03:27:20.720 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_list_pools PASSED [ 17%] 2026-02-26T03:27:20.740 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_fsid PASSED [ 18%] 2026-02-26T03:27:21.725 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_blocklist_add PASSED [ 19%] 2026-02-26T03:27:21.741 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_cluster_stats PASSED [ 20%] 2026-02-26T03:27:21.756 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_monitor_log PASSED [ 22%] 2026-02-26T03:27:24.735 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_last_version PASSED [ 23%] 2026-02-26T03:27:27.332 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_stats PASSED [ 24%] 2026-02-26T03:27:31.347 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write PASSED [ 25%] 2026-02-26T03:27:35.365 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_full PASSED [ 26%] 2026-02-26T03:27:39.381 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_writesame PASSED [ 27%] 2026-02-26T03:27:43.397 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_append PASSED [ 29%] 2026-02-26T03:27:47.413 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_zeros PASSED [ 30%] 2026-02-26T03:27:51.430 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_trunc PASSED [ 31%] 2026-02-26T03:27:55.443 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_cmpext PASSED [ 32%] 2026-02-26T03:27:59.463 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_objects_empty PASSED [ 33%] 2026-02-26T03:28:03.340 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_read_crc PASSED [ 34%] 2026-02-26T03:28:07.353 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_objects PASSED [ 36%] 2026-02-26T03:28:11.369 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_ns_objects PASSED [ 37%] 2026-02-26T03:28:15.335 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_xattrs PASSED [ 38%] 2026-02-26T03:28:19.351 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_obj_xattrs PASSED [ 39%] 2026-02-26T03:28:22.364 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_pool_id PASSED [ 40%] 2026-02-26T03:28:25.339 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_pool_name PASSED [ 41%] 2026-02-26T03:28:30.350 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_create_snap PASSED [ 43%] 2026-02-26T03:28:33.364 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_snaps_empty PASSED [ 44%] 2026-02-26T03:28:39.352 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_snaps PASSED [ 45%] 2026-02-26T03:28:43.369 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_lookup_snap PASSED [ 46%] 2026-02-26T03:28:47.344 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_timestamp PASSED [ 47%] 2026-02-26T03:28:52.363 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_remove_snap PASSED [ 48%] 2026-02-26T03:28:58.389 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_rollback PASSED [ 50%] 2026-02-26T03:29:04.409 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_rollback_removed PASSED [ 51%] 2026-02-26T03:29:10.358 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_read PASSED [ 52%] 2026-02-26T03:29:14.380 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_set_omap PASSED [ 53%] 2026-02-26T03:29:18.386 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_set_omap_aio PASSED [ 54%] 2026-02-26T03:29:22.400 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_ops PASSED [ 55%] 2026-02-26T03:29:26.352 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_execute_op PASSED [ 56%] 2026-02-26T03:29:30.363 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_writesame_op PASSED [ 58%] 2026-02-26T03:29:34.378 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_omap_vals_by_keys PASSED [ 59%] 2026-02-26T03:29:38.368 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_omap_keys PASSED [ 60%] 2026-02-26T03:29:42.382 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_clear_omap PASSED [ 61%] 2026-02-26T03:29:46.357 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_remove_omap_range2 PASSED [ 62%] 2026-02-26T03:29:50.365 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_omap_cmp PASSED [ 63%] 2026-02-26T03:29:54.385 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_cmpext_op PASSED [ 65%] 2026-02-26T03:29:58.393 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_xattrs_op PASSED [ 66%] 2026-02-26T03:30:01.405 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_locator PASSED [ 67%] 2026-02-26T03:30:05.360 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_operate_aio_write_op PASSED [ 68%] 2026-02-26T03:30:09.368 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write PASSED [ 69%] 2026-02-26T03:30:13.385 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_cmpext PASSED [ 70%] 2026-02-26T03:30:17.399 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_rmxattr PASSED [ 72%] 2026-02-26T03:30:21.415 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write_no_comp_ref PASSED [ 73%] 2026-02-26T03:30:25.361 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_append PASSED [ 74%] 2026-02-26T03:30:29.371 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write_full PASSED [ 75%] 2026-02-26T03:30:33.391 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_writesame PASSED [ 76%] 2026-02-26T03:30:37.403 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_stat PASSED [ 77%] 2026-02-26T03:30:41.420 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_remove PASSED [ 79%] 2026-02-26T03:30:45.367 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_lock PASSED [ 80%] 2026-02-26T03:30:49.376 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_execute PASSED [ 81%] 2026-02-26T03:30:53.392 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_execute PASSED [ 82%] 2026-02-26T03:30:57.405 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_setxattr PASSED [ 83%] 2026-02-26T03:31:06.373 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_applications PASSED [ 84%] 2026-02-26T03:31:09.381 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_service_daemon PASSED [ 86%] 2026-02-26T03:31:12.391 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_alignment PASSED [ 87%] 2026-02-26T03:31:15.375 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx2::test_get_last_version PASSED [ 88%] 2026-02-26T03:31:18.381 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx2::test_get_stats PASSED [ 89%] 2026-02-26T03:31:21.395 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_read PASSED [ 90%] 2026-02-26T03:31:24.408 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_seek PASSED [ 91%] 2026-02-26T03:31:27.409 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_write PASSED [ 93%] 2026-02-26T03:31:35.378 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoCtxSelfManagedSnaps::test PASSED [ 94%] 2026-02-26T03:31:35.396 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_monmap_dump PASSED [ 95%] 2026-02-26T03:31:35.458 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_osd_bench PASSED [ 96%] 2026-02-26T03:31:36.442 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_ceph_osd_pool_create_utf8 PASSED [ 97%] 2026-02-26T03:31:42.658 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestWatchNotify::test PASSED [ 98%] 2026-02-26T03:31:48.383 INFO:tasks.workunit.client.0.trial012.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestWatchNotify::test_aio_notify PASSED [100%] 2026-02-26T03:31:48.383 INFO:tasks.workunit.client.0.trial012.stdout: 2026-02-26T03:31:48.383 INFO:tasks.workunit.client.0.trial012.stdout:================= 86 passed, 5 deselected in 283.76s (0:04:43) ================= 2026-02-26T03:31:48.398 INFO:tasks.workunit.client.0.trial012.stderr:+ exit 0 2026-02-26T03:31:48.399 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-26T03:31:48.399 DEBUG:teuthology.orchestra.run.trial012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-26T03:31:48.469 INFO:tasks.workunit:Stopping ["rados/test_python.sh -m 'not (wait or tier or ec)'"] on client.0... 2026-02-26T03:31:48.469 DEBUG:teuthology.orchestra.run.trial012:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-26T03:31:48.861 DEBUG:teuthology.parallel:result is None 2026-02-26T03:31:48.861 DEBUG:teuthology.orchestra.run.trial012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-26T03:31:48.886 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-26T03:31:48.887 DEBUG:teuthology.orchestra.run.trial012:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-26T03:31:48.943 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-26T03:31:48.943 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-26T03:31:49.025 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-26T03:31:49.025 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:31:49.101 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:49.096+0000 7f6dab50b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:49.103 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:49.101+0000 7f6dab50b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:49.216 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:31:49.217 INFO:teuthology.orchestra.run.trial012.stderr:dumped all 2026-02-26T03:31:49.225 INFO:teuthology.orchestra.run.trial012.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2026-02-26T03:31:48.376214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":15336,"kb_used_data":15336,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208651288,"statfs":{"total":213674622976,"available":213658918912,"internally_reserved":0,"allocated":15704064,"data_stored":15704064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.779824"},"pg_stats":[{"pgid":"73.f","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669425+0000","last_change":"2026-02-26T03:31:37.534903+0000","last_active":"2026-02-26T03:31:43.669425+0000","last_peered":"2026-02-26T03:31:43.669425+0000","last_clean":"2026-02-26T03:31:43.669425+0000","last_became_active":"2026-02-26T03:31:37.512557+0000","last_became_peered":"2026-02-26T03:31:37.512557+0000","last_unstale":"2026-02-26T03:31:43.669425+0000","last_undegraded":"2026-02-26T03:31:43.669425+0000","last_fullsized":"2026-02-26T03:31:43.669425+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.e","version":"0'0","reported_seq":17,"reported_epoch":291,"state":"active+clean","last_fresh":"2026-02-26T03:31:42.671961+0000","last_change":"2026-02-26T03:31:37.496100+0000","last_active":"2026-02-26T03:31:42.671961+0000","last_peered":"2026-02-26T03:31:42.671961+0000","last_clean":"2026-02-26T03:31:42.671961+0000","last_became_active":"2026-02-26T03:31:37.489234+0000","last_became_peered":"2026-02-26T03:31:37.489234+0000","last_unstale":"2026-02-26T03:31:42.671961+0000","last_undegraded":"2026-02-26T03:31:42.671961+0000","last_fullsized":"2026-02-26T03:31:42.671961+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.d","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667593+0000","last_change":"2026-02-26T03:31:37.512242+0000","last_active":"2026-02-26T03:31:43.667593+0000","last_peered":"2026-02-26T03:31:43.667593+0000","last_clean":"2026-02-26T03:31:43.667593+0000","last_became_active":"2026-02-26T03:31:37.505462+0000","last_became_peered":"2026-02-26T03:31:37.505462+0000","last_unstale":"2026-02-26T03:31:43.667593+0000","last_undegraded":"2026-02-26T03:31:43.667593+0000","last_fullsized":"2026-02-26T03:31:43.667593+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.c","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668518+0000","last_change":"2026-02-26T03:31:37.508762+0000","last_active":"2026-02-26T03:31:43.668518+0000","last_peered":"2026-02-26T03:31:43.668518+0000","last_clean":"2026-02-26T03:31:43.668518+0000","last_became_active":"2026-02-26T03:31:37.495191+0000","last_became_peered":"2026-02-26T03:31:37.495191+0000","last_unstale":"2026-02-26T03:31:43.668518+0000","last_undegraded":"2026-02-26T03:31:43.668518+0000","last_fullsized":"2026-02-26T03:31:43.668518+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.b","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668948+0000","last_change":"2026-02-26T03:31:37.503079+0000","last_active":"2026-02-26T03:31:43.668948+0000","last_peered":"2026-02-26T03:31:43.668948+0000","last_clean":"2026-02-26T03:31:43.668948+0000","last_became_active":"2026-02-26T03:31:37.497201+0000","last_became_peered":"2026-02-26T03:31:37.497201+0000","last_unstale":"2026-02-26T03:31:43.668948+0000","last_undegraded":"2026-02-26T03:31:43.668948+0000","last_fullsized":"2026-02-26T03:31:43.668948+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.a","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668330+0000","last_change":"2026-02-26T03:31:37.527767+0000","last_active":"2026-02-26T03:31:43.668330+0000","last_peered":"2026-02-26T03:31:43.668330+0000","last_clean":"2026-02-26T03:31:43.668330+0000","last_became_active":"2026-02-26T03:31:37.506862+0000","last_became_peered":"2026-02-26T03:31:37.506862+0000","last_unstale":"2026-02-26T03:31:43.668330+0000","last_undegraded":"2026-02-26T03:31:43.668330+0000","last_fullsized":"2026-02-26T03:31:43.668330+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.9","version":"0'0","reported_seq":17,"reported_epoch":291,"state":"active+clean","last_fresh":"2026-02-26T03:31:42.666951+0000","last_change":"2026-02-26T03:31:37.496374+0000","last_active":"2026-02-26T03:31:42.666951+0000","last_peered":"2026-02-26T03:31:42.666951+0000","last_clean":"2026-02-26T03:31:42.666951+0000","last_became_active":"2026-02-26T03:31:37.489901+0000","last_became_peered":"2026-02-26T03:31:37.489901+0000","last_unstale":"2026-02-26T03:31:42.666951+0000","last_undegraded":"2026-02-26T03:31:42.666951+0000","last_fullsized":"2026-02-26T03:31:42.666951+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.8","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667817+0000","last_change":"2026-02-26T03:31:37.493086+0000","last_active":"2026-02-26T03:31:43.667817+0000","last_peered":"2026-02-26T03:31:43.667817+0000","last_clean":"2026-02-26T03:31:43.667817+0000","last_became_active":"2026-02-26T03:31:37.484153+0000","last_became_peered":"2026-02-26T03:31:37.484153+0000","last_unstale":"2026-02-26T03:31:43.667817+0000","last_undegraded":"2026-02-26T03:31:43.667817+0000","last_fullsized":"2026-02-26T03:31:43.667817+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.7","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669811+0000","last_change":"2026-02-26T03:31:37.553536+0000","last_active":"2026-02-26T03:31:43.669811+0000","last_peered":"2026-02-26T03:31:43.669811+0000","last_clean":"2026-02-26T03:31:43.669811+0000","last_became_active":"2026-02-26T03:31:37.522635+0000","last_became_peered":"2026-02-26T03:31:37.522635+0000","last_unstale":"2026-02-26T03:31:43.669811+0000","last_undegraded":"2026-02-26T03:31:43.669811+0000","last_fullsized":"2026-02-26T03:31:43.669811+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.6","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668191+0000","last_change":"2026-02-26T03:31:37.515418+0000","last_active":"2026-02-26T03:31:43.668191+0000","last_peered":"2026-02-26T03:31:43.668191+0000","last_clean":"2026-02-26T03:31:43.668191+0000","last_became_active":"2026-02-26T03:31:37.492882+0000","last_became_peered":"2026-02-26T03:31:37.492882+0000","last_unstale":"2026-02-26T03:31:43.668191+0000","last_undegraded":"2026-02-26T03:31:43.668191+0000","last_fullsized":"2026-02-26T03:31:43.668191+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.5","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669265+0000","last_change":"2026-02-26T03:31:37.551105+0000","last_active":"2026-02-26T03:31:43.669265+0000","last_peered":"2026-02-26T03:31:43.669265+0000","last_clean":"2026-02-26T03:31:43.669265+0000","last_became_active":"2026-02-26T03:31:37.544290+0000","last_became_peered":"2026-02-26T03:31:37.544290+0000","last_unstale":"2026-02-26T03:31:43.669265+0000","last_undegraded":"2026-02-26T03:31:43.669265+0000","last_fullsized":"2026-02-26T03:31:43.669265+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.4","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668743+0000","last_change":"2026-02-26T03:31:37.510764+0000","last_active":"2026-02-26T03:31:43.668743+0000","last_peered":"2026-02-26T03:31:43.668743+0000","last_clean":"2026-02-26T03:31:43.668743+0000","last_became_active":"2026-02-26T03:31:37.506074+0000","last_became_peered":"2026-02-26T03:31:37.506074+0000","last_unstale":"2026-02-26T03:31:43.668743+0000","last_undegraded":"2026-02-26T03:31:43.668743+0000","last_fullsized":"2026-02-26T03:31:43.668743+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.3","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668804+0000","last_change":"2026-02-26T03:31:37.561329+0000","last_active":"2026-02-26T03:31:43.668804+0000","last_peered":"2026-02-26T03:31:43.668804+0000","last_clean":"2026-02-26T03:31:43.668804+0000","last_became_active":"2026-02-26T03:31:37.554277+0000","last_became_peered":"2026-02-26T03:31:37.554277+0000","last_unstale":"2026-02-26T03:31:43.668804+0000","last_undegraded":"2026-02-26T03:31:43.668804+0000","last_fullsized":"2026-02-26T03:31:43.668804+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.2","version":"0'0","reported_seq":17,"reported_epoch":291,"state":"active+clean","last_fresh":"2026-02-26T03:31:42.672755+0000","last_change":"2026-02-26T03:31:37.524339+0000","last_active":"2026-02-26T03:31:42.672755+0000","last_peered":"2026-02-26T03:31:42.672755+0000","last_clean":"2026-02-26T03:31:42.672755+0000","last_became_active":"2026-02-26T03:31:37.518931+0000","last_became_peered":"2026-02-26T03:31:37.518931+0000","last_unstale":"2026-02-26T03:31:42.672755+0000","last_undegraded":"2026-02-26T03:31:42.672755+0000","last_fullsized":"2026-02-26T03:31:42.672755+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.1","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667964+0000","last_change":"2026-02-26T03:31:37.576337+0000","last_active":"2026-02-26T03:31:43.667964+0000","last_peered":"2026-02-26T03:31:43.667964+0000","last_clean":"2026-02-26T03:31:43.667964+0000","last_became_active":"2026-02-26T03:31:37.556206+0000","last_became_peered":"2026-02-26T03:31:37.556206+0000","last_unstale":"2026-02-26T03:31:43.667964+0000","last_undegraded":"2026-02-26T03:31:43.667964+0000","last_fullsized":"2026-02-26T03:31:43.667964+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.0","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667777+0000","last_change":"2026-02-26T03:31:37.507650+0000","last_active":"2026-02-26T03:31:43.667777+0000","last_peered":"2026-02-26T03:31:43.667777+0000","last_clean":"2026-02-26T03:31:43.667777+0000","last_became_active":"2026-02-26T03:31:37.493062+0000","last_became_peered":"2026-02-26T03:31:37.493062+0000","last_unstale":"2026-02-26T03:31:43.667777+0000","last_undegraded":"2026-02-26T03:31:43.667777+0000","last_fullsized":"2026-02-26T03:31:43.667777+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667643+0000","last_change":"2026-02-26T03:26:38.647558+0000","last_active":"2026-02-26T03:31:43.667643+0000","last_peered":"2026-02-26T03:31:43.667643+0000","last_clean":"2026-02-26T03:31:43.667643+0000","last_became_active":"2026-02-26T03:26:36.658652+0000","last_became_peered":"2026-02-26T03:26:36.658652+0000","last_unstale":"2026-02-26T03:31:43.667643+0000","last_undegraded":"2026-02-26T03:31:43.667643+0000","last_fullsized":"2026-02-26T03:31:43.667643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.670202+0000","last_change":"2026-02-26T03:26:38.648362+0000","last_active":"2026-02-26T03:31:43.670202+0000","last_peered":"2026-02-26T03:31:43.670202+0000","last_clean":"2026-02-26T03:31:43.670202+0000","last_became_active":"2026-02-26T03:26:36.669960+0000","last_became_peered":"2026-02-26T03:26:36.669960+0000","last_unstale":"2026-02-26T03:31:43.670202+0000","last_undegraded":"2026-02-26T03:31:43.670202+0000","last_fullsized":"2026-02-26T03:31:43.670202+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":848,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667345+0000","last_change":"2026-02-26T03:26:38.646292+0000","last_active":"2026-02-26T03:31:43.667345+0000","last_peered":"2026-02-26T03:31:43.667345+0000","last_clean":"2026-02-26T03:31:43.667345+0000","last_became_active":"2026-02-26T03:26:36.660207+0000","last_became_peered":"2026-02-26T03:26:36.660207+0000","last_unstale":"2026-02-26T03:31:43.667345+0000","last_undegraded":"2026-02-26T03:31:43.667345+0000","last_fullsized":"2026-02-26T03:31:43.667345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":844,"reported_epoch":291,"state":"active+clean","last_fresh":"2026-02-26T03:31:42.669132+0000","last_change":"2026-02-26T03:26:38.645479+0000","last_active":"2026-02-26T03:31:42.669132+0000","last_peered":"2026-02-26T03:31:42.669132+0000","last_clean":"2026-02-26T03:31:42.669132+0000","last_became_active":"2026-02-26T03:26:36.666405+0000","last_became_peered":"2026-02-26T03:26:36.666405+0000","last_unstale":"2026-02-26T03:31:42.669132+0000","last_undegraded":"2026-02-26T03:31:42.669132+0000","last_fullsized":"2026-02-26T03:31:42.669132+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":912,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669698+0000","last_change":"2026-02-26T03:26:36.680586+0000","last_active":"2026-02-26T03:31:43.669698+0000","last_peered":"2026-02-26T03:31:43.669698+0000","last_clean":"2026-02-26T03:31:43.669698+0000","last_became_active":"2026-02-26T03:26:36.667237+0000","last_became_peered":"2026-02-26T03:26:36.667237+0000","last_unstale":"2026-02-26T03:31:43.669698+0000","last_undegraded":"2026-02-26T03:31:43.669698+0000","last_fullsized":"2026-02-26T03:31:43.669698+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":843,"reported_epoch":291,"state":"active+clean","last_fresh":"2026-02-26T03:31:42.670472+0000","last_change":"2026-02-26T03:26:38.646875+0000","last_active":"2026-02-26T03:31:42.670472+0000","last_peered":"2026-02-26T03:31:42.670472+0000","last_clean":"2026-02-26T03:31:42.670472+0000","last_became_active":"2026-02-26T03:26:36.662567+0000","last_became_peered":"2026-02-26T03:26:36.662567+0000","last_unstale":"2026-02-26T03:31:42.670472+0000","last_undegraded":"2026-02-26T03:31:42.670472+0000","last_fullsized":"2026-02-26T03:31:42.670472+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669601+0000","last_change":"2026-02-26T03:26:38.647940+0000","last_active":"2026-02-26T03:31:43.669601+0000","last_peered":"2026-02-26T03:31:43.669601+0000","last_clean":"2026-02-26T03:31:43.669601+0000","last_became_active":"2026-02-26T03:26:36.672454+0000","last_became_peered":"2026-02-26T03:26:36.672454+0000","last_unstale":"2026-02-26T03:31:43.669601+0000","last_undegraded":"2026-02-26T03:31:43.669601+0000","last_fullsized":"2026-02-26T03:31:43.669601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668975+0000","last_change":"2026-02-26T03:26:38.647886+0000","last_active":"2026-02-26T03:31:43.668975+0000","last_peered":"2026-02-26T03:31:43.668975+0000","last_clean":"2026-02-26T03:31:43.668975+0000","last_became_active":"2026-02-26T03:26:36.667431+0000","last_became_peered":"2026-02-26T03:26:36.667431+0000","last_unstale":"2026-02-26T03:31:43.668975+0000","last_undegraded":"2026-02-26T03:31:43.668975+0000","last_fullsized":"2026-02-26T03:31:43.668975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668583+0000","last_change":"2026-02-26T03:26:38.647488+0000","last_active":"2026-02-26T03:31:43.668583+0000","last_peered":"2026-02-26T03:31:43.668583+0000","last_clean":"2026-02-26T03:31:43.668583+0000","last_became_active":"2026-02-26T03:26:36.663643+0000","last_became_peered":"2026-02-26T03:26:36.663643+0000","last_unstale":"2026-02-26T03:31:43.668583+0000","last_undegraded":"2026-02-26T03:31:43.668583+0000","last_fullsized":"2026-02-26T03:31:43.668583+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":73,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":213674622976,"available":213658886144,"internally_reserved":0,"allocated":15736832,"data_stored":15736832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160243855360,"internally_reserved":0,"allocated":12111872,"data_stored":12111872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":4180,"kb_used_data":4180,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162476,"statfs":{"total":53418655744,"available":53414375424,"internally_reserved":0,"allocated":4280320,"data_stored":4280320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3540,"kb_used_data":3540,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163116,"statfs":{"total":53418655744,"available":53415030784,"internally_reserved":0,"allocated":3624960,"data_stored":3624960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":4124,"kb_used_data":4124,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162532,"statfs":{"total":53418655744,"available":53414432768,"internally_reserved":0,"allocated":4222976,"data_stored":4222976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3492,"kb_used_data":3492,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163164,"statfs":{"total":53418655744,"available":53415079936,"internally_reserved":0,"allocated":3575808,"data_stored":3575808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53414580224,"internally_reserved":0,"allocated":4075520,"data_stored":4075520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53415014400,"internally_reserved":0,"allocated":3641344,"data_stored":3641344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":0,"total":53418655744,"available":53415030784,"internally_reserved":0,"allocated":3624960,"data_stored":3624960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":1,"total":53418655744,"available":53414580224,"internally_reserved":0,"allocated":4075520,"data_stored":4075520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":2,"total":53418655744,"available":53415014400,"internally_reserved":0,"allocated":3641344,"data_stored":3641344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":3,"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:31:49.228 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:31:49.302 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:49.298+0000 7ffaf170b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:49.305 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:49.302+0000 7ffaf170b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:49.417 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:31:49.418 INFO:teuthology.orchestra.run.trial012.stderr:dumped all 2026-02-26T03:31:49.427 INFO:teuthology.orchestra.run.trial012.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2026-02-26T03:31:48.376214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":15336,"kb_used_data":15336,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208651288,"statfs":{"total":213674622976,"available":213658918912,"internally_reserved":0,"allocated":15704064,"data_stored":15704064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.779824"},"pg_stats":[{"pgid":"73.f","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669425+0000","last_change":"2026-02-26T03:31:37.534903+0000","last_active":"2026-02-26T03:31:43.669425+0000","last_peered":"2026-02-26T03:31:43.669425+0000","last_clean":"2026-02-26T03:31:43.669425+0000","last_became_active":"2026-02-26T03:31:37.512557+0000","last_became_peered":"2026-02-26T03:31:37.512557+0000","last_unstale":"2026-02-26T03:31:43.669425+0000","last_undegraded":"2026-02-26T03:31:43.669425+0000","last_fullsized":"2026-02-26T03:31:43.669425+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.e","version":"0'0","reported_seq":17,"reported_epoch":291,"state":"active+clean","last_fresh":"2026-02-26T03:31:42.671961+0000","last_change":"2026-02-26T03:31:37.496100+0000","last_active":"2026-02-26T03:31:42.671961+0000","last_peered":"2026-02-26T03:31:42.671961+0000","last_clean":"2026-02-26T03:31:42.671961+0000","last_became_active":"2026-02-26T03:31:37.489234+0000","last_became_peered":"2026-02-26T03:31:37.489234+0000","last_unstale":"2026-02-26T03:31:42.671961+0000","last_undegraded":"2026-02-26T03:31:42.671961+0000","last_fullsized":"2026-02-26T03:31:42.671961+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.d","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667593+0000","last_change":"2026-02-26T03:31:37.512242+0000","last_active":"2026-02-26T03:31:43.667593+0000","last_peered":"2026-02-26T03:31:43.667593+0000","last_clean":"2026-02-26T03:31:43.667593+0000","last_became_active":"2026-02-26T03:31:37.505462+0000","last_became_peered":"2026-02-26T03:31:37.505462+0000","last_unstale":"2026-02-26T03:31:43.667593+0000","last_undegraded":"2026-02-26T03:31:43.667593+0000","last_fullsized":"2026-02-26T03:31:43.667593+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.c","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668518+0000","last_change":"2026-02-26T03:31:37.508762+0000","last_active":"2026-02-26T03:31:43.668518+0000","last_peered":"2026-02-26T03:31:43.668518+0000","last_clean":"2026-02-26T03:31:43.668518+0000","last_became_active":"2026-02-26T03:31:37.495191+0000","last_became_peered":"2026-02-26T03:31:37.495191+0000","last_unstale":"2026-02-26T03:31:43.668518+0000","last_undegraded":"2026-02-26T03:31:43.668518+0000","last_fullsized":"2026-02-26T03:31:43.668518+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.b","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668948+0000","last_change":"2026-02-26T03:31:37.503079+0000","last_active":"2026-02-26T03:31:43.668948+0000","last_peered":"2026-02-26T03:31:43.668948+0000","last_clean":"2026-02-26T03:31:43.668948+0000","last_became_active":"2026-02-26T03:31:37.497201+0000","last_became_peered":"2026-02-26T03:31:37.497201+0000","last_unstale":"2026-02-26T03:31:43.668948+0000","last_undegraded":"2026-02-26T03:31:43.668948+0000","last_fullsized":"2026-02-26T03:31:43.668948+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.a","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668330+0000","last_change":"2026-02-26T03:31:37.527767+0000","last_active":"2026-02-26T03:31:43.668330+0000","last_peered":"2026-02-26T03:31:43.668330+0000","last_clean":"2026-02-26T03:31:43.668330+0000","last_became_active":"2026-02-26T03:31:37.506862+0000","last_became_peered":"2026-02-26T03:31:37.506862+0000","last_unstale":"2026-02-26T03:31:43.668330+0000","last_undegraded":"2026-02-26T03:31:43.668330+0000","last_fullsized":"2026-02-26T03:31:43.668330+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.9","version":"0'0","reported_seq":17,"reported_epoch":291,"state":"active+clean","last_fresh":"2026-02-26T03:31:42.666951+0000","last_change":"2026-02-26T03:31:37.496374+0000","last_active":"2026-02-26T03:31:42.666951+0000","last_peered":"2026-02-26T03:31:42.666951+0000","last_clean":"2026-02-26T03:31:42.666951+0000","last_became_active":"2026-02-26T03:31:37.489901+0000","last_became_peered":"2026-02-26T03:31:37.489901+0000","last_unstale":"2026-02-26T03:31:42.666951+0000","last_undegraded":"2026-02-26T03:31:42.666951+0000","last_fullsized":"2026-02-26T03:31:42.666951+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.8","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667817+0000","last_change":"2026-02-26T03:31:37.493086+0000","last_active":"2026-02-26T03:31:43.667817+0000","last_peered":"2026-02-26T03:31:43.667817+0000","last_clean":"2026-02-26T03:31:43.667817+0000","last_became_active":"2026-02-26T03:31:37.484153+0000","last_became_peered":"2026-02-26T03:31:37.484153+0000","last_unstale":"2026-02-26T03:31:43.667817+0000","last_undegraded":"2026-02-26T03:31:43.667817+0000","last_fullsized":"2026-02-26T03:31:43.667817+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.7","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669811+0000","last_change":"2026-02-26T03:31:37.553536+0000","last_active":"2026-02-26T03:31:43.669811+0000","last_peered":"2026-02-26T03:31:43.669811+0000","last_clean":"2026-02-26T03:31:43.669811+0000","last_became_active":"2026-02-26T03:31:37.522635+0000","last_became_peered":"2026-02-26T03:31:37.522635+0000","last_unstale":"2026-02-26T03:31:43.669811+0000","last_undegraded":"2026-02-26T03:31:43.669811+0000","last_fullsized":"2026-02-26T03:31:43.669811+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.6","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668191+0000","last_change":"2026-02-26T03:31:37.515418+0000","last_active":"2026-02-26T03:31:43.668191+0000","last_peered":"2026-02-26T03:31:43.668191+0000","last_clean":"2026-02-26T03:31:43.668191+0000","last_became_active":"2026-02-26T03:31:37.492882+0000","last_became_peered":"2026-02-26T03:31:37.492882+0000","last_unstale":"2026-02-26T03:31:43.668191+0000","last_undegraded":"2026-02-26T03:31:43.668191+0000","last_fullsized":"2026-02-26T03:31:43.668191+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.5","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669265+0000","last_change":"2026-02-26T03:31:37.551105+0000","last_active":"2026-02-26T03:31:43.669265+0000","last_peered":"2026-02-26T03:31:43.669265+0000","last_clean":"2026-02-26T03:31:43.669265+0000","last_became_active":"2026-02-26T03:31:37.544290+0000","last_became_peered":"2026-02-26T03:31:37.544290+0000","last_unstale":"2026-02-26T03:31:43.669265+0000","last_undegraded":"2026-02-26T03:31:43.669265+0000","last_fullsized":"2026-02-26T03:31:43.669265+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.4","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668743+0000","last_change":"2026-02-26T03:31:37.510764+0000","last_active":"2026-02-26T03:31:43.668743+0000","last_peered":"2026-02-26T03:31:43.668743+0000","last_clean":"2026-02-26T03:31:43.668743+0000","last_became_active":"2026-02-26T03:31:37.506074+0000","last_became_peered":"2026-02-26T03:31:37.506074+0000","last_unstale":"2026-02-26T03:31:43.668743+0000","last_undegraded":"2026-02-26T03:31:43.668743+0000","last_fullsized":"2026-02-26T03:31:43.668743+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.3","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668804+0000","last_change":"2026-02-26T03:31:37.561329+0000","last_active":"2026-02-26T03:31:43.668804+0000","last_peered":"2026-02-26T03:31:43.668804+0000","last_clean":"2026-02-26T03:31:43.668804+0000","last_became_active":"2026-02-26T03:31:37.554277+0000","last_became_peered":"2026-02-26T03:31:37.554277+0000","last_unstale":"2026-02-26T03:31:43.668804+0000","last_undegraded":"2026-02-26T03:31:43.668804+0000","last_fullsized":"2026-02-26T03:31:43.668804+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.2","version":"0'0","reported_seq":17,"reported_epoch":291,"state":"active+clean","last_fresh":"2026-02-26T03:31:42.672755+0000","last_change":"2026-02-26T03:31:37.524339+0000","last_active":"2026-02-26T03:31:42.672755+0000","last_peered":"2026-02-26T03:31:42.672755+0000","last_clean":"2026-02-26T03:31:42.672755+0000","last_became_active":"2026-02-26T03:31:37.518931+0000","last_became_peered":"2026-02-26T03:31:37.518931+0000","last_unstale":"2026-02-26T03:31:42.672755+0000","last_undegraded":"2026-02-26T03:31:42.672755+0000","last_fullsized":"2026-02-26T03:31:42.672755+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.1","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667964+0000","last_change":"2026-02-26T03:31:37.576337+0000","last_active":"2026-02-26T03:31:43.667964+0000","last_peered":"2026-02-26T03:31:43.667964+0000","last_clean":"2026-02-26T03:31:43.667964+0000","last_became_active":"2026-02-26T03:31:37.556206+0000","last_became_peered":"2026-02-26T03:31:37.556206+0000","last_unstale":"2026-02-26T03:31:43.667964+0000","last_undegraded":"2026-02-26T03:31:43.667964+0000","last_fullsized":"2026-02-26T03:31:43.667964+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.0","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667777+0000","last_change":"2026-02-26T03:31:37.507650+0000","last_active":"2026-02-26T03:31:43.667777+0000","last_peered":"2026-02-26T03:31:43.667777+0000","last_clean":"2026-02-26T03:31:43.667777+0000","last_became_active":"2026-02-26T03:31:37.493062+0000","last_became_peered":"2026-02-26T03:31:37.493062+0000","last_unstale":"2026-02-26T03:31:43.667777+0000","last_undegraded":"2026-02-26T03:31:43.667777+0000","last_fullsized":"2026-02-26T03:31:43.667777+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667643+0000","last_change":"2026-02-26T03:26:38.647558+0000","last_active":"2026-02-26T03:31:43.667643+0000","last_peered":"2026-02-26T03:31:43.667643+0000","last_clean":"2026-02-26T03:31:43.667643+0000","last_became_active":"2026-02-26T03:26:36.658652+0000","last_became_peered":"2026-02-26T03:26:36.658652+0000","last_unstale":"2026-02-26T03:31:43.667643+0000","last_undegraded":"2026-02-26T03:31:43.667643+0000","last_fullsized":"2026-02-26T03:31:43.667643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.670202+0000","last_change":"2026-02-26T03:26:38.648362+0000","last_active":"2026-02-26T03:31:43.670202+0000","last_peered":"2026-02-26T03:31:43.670202+0000","last_clean":"2026-02-26T03:31:43.670202+0000","last_became_active":"2026-02-26T03:26:36.669960+0000","last_became_peered":"2026-02-26T03:26:36.669960+0000","last_unstale":"2026-02-26T03:31:43.670202+0000","last_undegraded":"2026-02-26T03:31:43.670202+0000","last_fullsized":"2026-02-26T03:31:43.670202+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":848,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667345+0000","last_change":"2026-02-26T03:26:38.646292+0000","last_active":"2026-02-26T03:31:43.667345+0000","last_peered":"2026-02-26T03:31:43.667345+0000","last_clean":"2026-02-26T03:31:43.667345+0000","last_became_active":"2026-02-26T03:26:36.660207+0000","last_became_peered":"2026-02-26T03:26:36.660207+0000","last_unstale":"2026-02-26T03:31:43.667345+0000","last_undegraded":"2026-02-26T03:31:43.667345+0000","last_fullsized":"2026-02-26T03:31:43.667345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":844,"reported_epoch":291,"state":"active+clean","last_fresh":"2026-02-26T03:31:42.669132+0000","last_change":"2026-02-26T03:26:38.645479+0000","last_active":"2026-02-26T03:31:42.669132+0000","last_peered":"2026-02-26T03:31:42.669132+0000","last_clean":"2026-02-26T03:31:42.669132+0000","last_became_active":"2026-02-26T03:26:36.666405+0000","last_became_peered":"2026-02-26T03:26:36.666405+0000","last_unstale":"2026-02-26T03:31:42.669132+0000","last_undegraded":"2026-02-26T03:31:42.669132+0000","last_fullsized":"2026-02-26T03:31:42.669132+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":912,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669698+0000","last_change":"2026-02-26T03:26:36.680586+0000","last_active":"2026-02-26T03:31:43.669698+0000","last_peered":"2026-02-26T03:31:43.669698+0000","last_clean":"2026-02-26T03:31:43.669698+0000","last_became_active":"2026-02-26T03:26:36.667237+0000","last_became_peered":"2026-02-26T03:26:36.667237+0000","last_unstale":"2026-02-26T03:31:43.669698+0000","last_undegraded":"2026-02-26T03:31:43.669698+0000","last_fullsized":"2026-02-26T03:31:43.669698+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":843,"reported_epoch":291,"state":"active+clean","last_fresh":"2026-02-26T03:31:42.670472+0000","last_change":"2026-02-26T03:26:38.646875+0000","last_active":"2026-02-26T03:31:42.670472+0000","last_peered":"2026-02-26T03:31:42.670472+0000","last_clean":"2026-02-26T03:31:42.670472+0000","last_became_active":"2026-02-26T03:26:36.662567+0000","last_became_peered":"2026-02-26T03:26:36.662567+0000","last_unstale":"2026-02-26T03:31:42.670472+0000","last_undegraded":"2026-02-26T03:31:42.670472+0000","last_fullsized":"2026-02-26T03:31:42.670472+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669601+0000","last_change":"2026-02-26T03:26:38.647940+0000","last_active":"2026-02-26T03:31:43.669601+0000","last_peered":"2026-02-26T03:31:43.669601+0000","last_clean":"2026-02-26T03:31:43.669601+0000","last_became_active":"2026-02-26T03:26:36.672454+0000","last_became_peered":"2026-02-26T03:26:36.672454+0000","last_unstale":"2026-02-26T03:31:43.669601+0000","last_undegraded":"2026-02-26T03:31:43.669601+0000","last_fullsized":"2026-02-26T03:31:43.669601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668975+0000","last_change":"2026-02-26T03:26:38.647886+0000","last_active":"2026-02-26T03:31:43.668975+0000","last_peered":"2026-02-26T03:31:43.668975+0000","last_clean":"2026-02-26T03:31:43.668975+0000","last_became_active":"2026-02-26T03:26:36.667431+0000","last_became_peered":"2026-02-26T03:26:36.667431+0000","last_unstale":"2026-02-26T03:31:43.668975+0000","last_undegraded":"2026-02-26T03:31:43.668975+0000","last_fullsized":"2026-02-26T03:31:43.668975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668583+0000","last_change":"2026-02-26T03:26:38.647488+0000","last_active":"2026-02-26T03:31:43.668583+0000","last_peered":"2026-02-26T03:31:43.668583+0000","last_clean":"2026-02-26T03:31:43.668583+0000","last_became_active":"2026-02-26T03:26:36.663643+0000","last_became_peered":"2026-02-26T03:26:36.663643+0000","last_unstale":"2026-02-26T03:31:43.668583+0000","last_undegraded":"2026-02-26T03:31:43.668583+0000","last_fullsized":"2026-02-26T03:31:43.668583+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":73,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":213674622976,"available":213658886144,"internally_reserved":0,"allocated":15736832,"data_stored":15736832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160243855360,"internally_reserved":0,"allocated":12111872,"data_stored":12111872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":4180,"kb_used_data":4180,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162476,"statfs":{"total":53418655744,"available":53414375424,"internally_reserved":0,"allocated":4280320,"data_stored":4280320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3540,"kb_used_data":3540,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163116,"statfs":{"total":53418655744,"available":53415030784,"internally_reserved":0,"allocated":3624960,"data_stored":3624960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":4124,"kb_used_data":4124,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162532,"statfs":{"total":53418655744,"available":53414432768,"internally_reserved":0,"allocated":4222976,"data_stored":4222976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3492,"kb_used_data":3492,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163164,"statfs":{"total":53418655744,"available":53415079936,"internally_reserved":0,"allocated":3575808,"data_stored":3575808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53414580224,"internally_reserved":0,"allocated":4075520,"data_stored":4075520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53415014400,"internally_reserved":0,"allocated":3641344,"data_stored":3641344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":0,"total":53418655744,"available":53415030784,"internally_reserved":0,"allocated":3624960,"data_stored":3624960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":1,"total":53418655744,"available":53414580224,"internally_reserved":0,"allocated":4075520,"data_stored":4075520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":2,"total":53418655744,"available":53415014400,"internally_reserved":0,"allocated":3641344,"data_stored":3641344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":3,"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:31:49.430 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-26T03:31:49.430 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:31:49.507 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:49.502+0000 7f41bba5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:49.510 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:49.508+0000 7f41bba5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:49.621 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:31:49.622 INFO:teuthology.orchestra.run.trial012.stderr:dumped all 2026-02-26T03:31:49.630 INFO:teuthology.orchestra.run.trial012.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2026-02-26T03:31:49.597245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":15360,"kb_used_data":15360,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208651264,"statfs":{"total":213674622976,"available":213658894336,"internally_reserved":0,"allocated":15728640,"data_stored":15728640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.942902"},"pg_stats":[{"pgid":"73.f","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669425+0000","last_change":"2026-02-26T03:31:37.534903+0000","last_active":"2026-02-26T03:31:43.669425+0000","last_peered":"2026-02-26T03:31:43.669425+0000","last_clean":"2026-02-26T03:31:43.669425+0000","last_became_active":"2026-02-26T03:31:37.512557+0000","last_became_peered":"2026-02-26T03:31:37.512557+0000","last_unstale":"2026-02-26T03:31:43.669425+0000","last_undegraded":"2026-02-26T03:31:43.669425+0000","last_fullsized":"2026-02-26T03:31:43.669425+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.e","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.403682+0000","last_change":"2026-02-26T03:31:37.496100+0000","last_active":"2026-02-26T03:31:48.403682+0000","last_peered":"2026-02-26T03:31:48.403682+0000","last_clean":"2026-02-26T03:31:48.403682+0000","last_became_active":"2026-02-26T03:31:37.489234+0000","last_became_peered":"2026-02-26T03:31:37.489234+0000","last_unstale":"2026-02-26T03:31:48.403682+0000","last_undegraded":"2026-02-26T03:31:48.403682+0000","last_fullsized":"2026-02-26T03:31:48.403682+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.d","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667593+0000","last_change":"2026-02-26T03:31:37.512242+0000","last_active":"2026-02-26T03:31:43.667593+0000","last_peered":"2026-02-26T03:31:43.667593+0000","last_clean":"2026-02-26T03:31:43.667593+0000","last_became_active":"2026-02-26T03:31:37.505462+0000","last_became_peered":"2026-02-26T03:31:37.505462+0000","last_unstale":"2026-02-26T03:31:43.667593+0000","last_undegraded":"2026-02-26T03:31:43.667593+0000","last_fullsized":"2026-02-26T03:31:43.667593+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.c","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668518+0000","last_change":"2026-02-26T03:31:37.508762+0000","last_active":"2026-02-26T03:31:43.668518+0000","last_peered":"2026-02-26T03:31:43.668518+0000","last_clean":"2026-02-26T03:31:43.668518+0000","last_became_active":"2026-02-26T03:31:37.495191+0000","last_became_peered":"2026-02-26T03:31:37.495191+0000","last_unstale":"2026-02-26T03:31:43.668518+0000","last_undegraded":"2026-02-26T03:31:43.668518+0000","last_fullsized":"2026-02-26T03:31:43.668518+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.b","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668948+0000","last_change":"2026-02-26T03:31:37.503079+0000","last_active":"2026-02-26T03:31:43.668948+0000","last_peered":"2026-02-26T03:31:43.668948+0000","last_clean":"2026-02-26T03:31:43.668948+0000","last_became_active":"2026-02-26T03:31:37.497201+0000","last_became_peered":"2026-02-26T03:31:37.497201+0000","last_unstale":"2026-02-26T03:31:43.668948+0000","last_undegraded":"2026-02-26T03:31:43.668948+0000","last_fullsized":"2026-02-26T03:31:43.668948+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.a","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668330+0000","last_change":"2026-02-26T03:31:37.527767+0000","last_active":"2026-02-26T03:31:43.668330+0000","last_peered":"2026-02-26T03:31:43.668330+0000","last_clean":"2026-02-26T03:31:43.668330+0000","last_became_active":"2026-02-26T03:31:37.506862+0000","last_became_peered":"2026-02-26T03:31:37.506862+0000","last_unstale":"2026-02-26T03:31:43.668330+0000","last_undegraded":"2026-02-26T03:31:43.668330+0000","last_fullsized":"2026-02-26T03:31:43.668330+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.9","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.403208+0000","last_change":"2026-02-26T03:31:37.496374+0000","last_active":"2026-02-26T03:31:48.403208+0000","last_peered":"2026-02-26T03:31:48.403208+0000","last_clean":"2026-02-26T03:31:48.403208+0000","last_became_active":"2026-02-26T03:31:37.489901+0000","last_became_peered":"2026-02-26T03:31:37.489901+0000","last_unstale":"2026-02-26T03:31:48.403208+0000","last_undegraded":"2026-02-26T03:31:48.403208+0000","last_fullsized":"2026-02-26T03:31:48.403208+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.8","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667817+0000","last_change":"2026-02-26T03:31:37.493086+0000","last_active":"2026-02-26T03:31:43.667817+0000","last_peered":"2026-02-26T03:31:43.667817+0000","last_clean":"2026-02-26T03:31:43.667817+0000","last_became_active":"2026-02-26T03:31:37.484153+0000","last_became_peered":"2026-02-26T03:31:37.484153+0000","last_unstale":"2026-02-26T03:31:43.667817+0000","last_undegraded":"2026-02-26T03:31:43.667817+0000","last_fullsized":"2026-02-26T03:31:43.667817+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.7","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669811+0000","last_change":"2026-02-26T03:31:37.553536+0000","last_active":"2026-02-26T03:31:43.669811+0000","last_peered":"2026-02-26T03:31:43.669811+0000","last_clean":"2026-02-26T03:31:43.669811+0000","last_became_active":"2026-02-26T03:31:37.522635+0000","last_became_peered":"2026-02-26T03:31:37.522635+0000","last_unstale":"2026-02-26T03:31:43.669811+0000","last_undegraded":"2026-02-26T03:31:43.669811+0000","last_fullsized":"2026-02-26T03:31:43.669811+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.6","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668191+0000","last_change":"2026-02-26T03:31:37.515418+0000","last_active":"2026-02-26T03:31:43.668191+0000","last_peered":"2026-02-26T03:31:43.668191+0000","last_clean":"2026-02-26T03:31:43.668191+0000","last_became_active":"2026-02-26T03:31:37.492882+0000","last_became_peered":"2026-02-26T03:31:37.492882+0000","last_unstale":"2026-02-26T03:31:43.668191+0000","last_undegraded":"2026-02-26T03:31:43.668191+0000","last_fullsized":"2026-02-26T03:31:43.668191+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.5","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669265+0000","last_change":"2026-02-26T03:31:37.551105+0000","last_active":"2026-02-26T03:31:43.669265+0000","last_peered":"2026-02-26T03:31:43.669265+0000","last_clean":"2026-02-26T03:31:43.669265+0000","last_became_active":"2026-02-26T03:31:37.544290+0000","last_became_peered":"2026-02-26T03:31:37.544290+0000","last_unstale":"2026-02-26T03:31:43.669265+0000","last_undegraded":"2026-02-26T03:31:43.669265+0000","last_fullsized":"2026-02-26T03:31:43.669265+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.4","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668743+0000","last_change":"2026-02-26T03:31:37.510764+0000","last_active":"2026-02-26T03:31:43.668743+0000","last_peered":"2026-02-26T03:31:43.668743+0000","last_clean":"2026-02-26T03:31:43.668743+0000","last_became_active":"2026-02-26T03:31:37.506074+0000","last_became_peered":"2026-02-26T03:31:37.506074+0000","last_unstale":"2026-02-26T03:31:43.668743+0000","last_undegraded":"2026-02-26T03:31:43.668743+0000","last_fullsized":"2026-02-26T03:31:43.668743+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.3","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668804+0000","last_change":"2026-02-26T03:31:37.561329+0000","last_active":"2026-02-26T03:31:43.668804+0000","last_peered":"2026-02-26T03:31:43.668804+0000","last_clean":"2026-02-26T03:31:43.668804+0000","last_became_active":"2026-02-26T03:31:37.554277+0000","last_became_peered":"2026-02-26T03:31:37.554277+0000","last_unstale":"2026-02-26T03:31:43.668804+0000","last_undegraded":"2026-02-26T03:31:43.668804+0000","last_fullsized":"2026-02-26T03:31:43.668804+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.2","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.404442+0000","last_change":"2026-02-26T03:31:37.524339+0000","last_active":"2026-02-26T03:31:48.404442+0000","last_peered":"2026-02-26T03:31:48.404442+0000","last_clean":"2026-02-26T03:31:48.404442+0000","last_became_active":"2026-02-26T03:31:37.518931+0000","last_became_peered":"2026-02-26T03:31:37.518931+0000","last_unstale":"2026-02-26T03:31:48.404442+0000","last_undegraded":"2026-02-26T03:31:48.404442+0000","last_fullsized":"2026-02-26T03:31:48.404442+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.1","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667964+0000","last_change":"2026-02-26T03:31:37.576337+0000","last_active":"2026-02-26T03:31:43.667964+0000","last_peered":"2026-02-26T03:31:43.667964+0000","last_clean":"2026-02-26T03:31:43.667964+0000","last_became_active":"2026-02-26T03:31:37.556206+0000","last_became_peered":"2026-02-26T03:31:37.556206+0000","last_unstale":"2026-02-26T03:31:43.667964+0000","last_undegraded":"2026-02-26T03:31:43.667964+0000","last_fullsized":"2026-02-26T03:31:43.667964+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.0","version":"0'0","reported_seq":20,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667777+0000","last_change":"2026-02-26T03:31:37.507650+0000","last_active":"2026-02-26T03:31:43.667777+0000","last_peered":"2026-02-26T03:31:43.667777+0000","last_clean":"2026-02-26T03:31:43.667777+0000","last_became_active":"2026-02-26T03:31:37.493062+0000","last_became_peered":"2026-02-26T03:31:37.493062+0000","last_unstale":"2026-02-26T03:31:43.667777+0000","last_undegraded":"2026-02-26T03:31:43.667777+0000","last_fullsized":"2026-02-26T03:31:43.667777+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667643+0000","last_change":"2026-02-26T03:26:38.647558+0000","last_active":"2026-02-26T03:31:43.667643+0000","last_peered":"2026-02-26T03:31:43.667643+0000","last_clean":"2026-02-26T03:31:43.667643+0000","last_became_active":"2026-02-26T03:26:36.658652+0000","last_became_peered":"2026-02-26T03:26:36.658652+0000","last_unstale":"2026-02-26T03:31:43.667643+0000","last_undegraded":"2026-02-26T03:31:43.667643+0000","last_fullsized":"2026-02-26T03:31:43.667643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.670202+0000","last_change":"2026-02-26T03:26:38.648362+0000","last_active":"2026-02-26T03:31:43.670202+0000","last_peered":"2026-02-26T03:31:43.670202+0000","last_clean":"2026-02-26T03:31:43.670202+0000","last_became_active":"2026-02-26T03:26:36.669960+0000","last_became_peered":"2026-02-26T03:26:36.669960+0000","last_unstale":"2026-02-26T03:31:43.670202+0000","last_undegraded":"2026-02-26T03:31:43.670202+0000","last_fullsized":"2026-02-26T03:31:43.670202+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":848,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.667345+0000","last_change":"2026-02-26T03:26:38.646292+0000","last_active":"2026-02-26T03:31:43.667345+0000","last_peered":"2026-02-26T03:31:43.667345+0000","last_clean":"2026-02-26T03:31:43.667345+0000","last_became_active":"2026-02-26T03:26:36.660207+0000","last_became_peered":"2026-02-26T03:26:36.660207+0000","last_unstale":"2026-02-26T03:31:43.667345+0000","last_undegraded":"2026-02-26T03:31:43.667345+0000","last_fullsized":"2026-02-26T03:31:43.667345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":856,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.398292+0000","last_change":"2026-02-26T03:26:38.645479+0000","last_active":"2026-02-26T03:31:48.398292+0000","last_peered":"2026-02-26T03:31:48.398292+0000","last_clean":"2026-02-26T03:31:48.398292+0000","last_became_active":"2026-02-26T03:26:36.666405+0000","last_became_peered":"2026-02-26T03:26:36.666405+0000","last_unstale":"2026-02-26T03:31:48.398292+0000","last_undegraded":"2026-02-26T03:31:48.398292+0000","last_fullsized":"2026-02-26T03:31:48.398292+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":912,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669698+0000","last_change":"2026-02-26T03:26:36.680586+0000","last_active":"2026-02-26T03:31:43.669698+0000","last_peered":"2026-02-26T03:31:43.669698+0000","last_clean":"2026-02-26T03:31:43.669698+0000","last_became_active":"2026-02-26T03:26:36.667237+0000","last_became_peered":"2026-02-26T03:26:36.667237+0000","last_unstale":"2026-02-26T03:31:43.669698+0000","last_undegraded":"2026-02-26T03:31:43.669698+0000","last_fullsized":"2026-02-26T03:31:43.669698+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.405807+0000","last_change":"2026-02-26T03:26:38.646875+0000","last_active":"2026-02-26T03:31:48.405807+0000","last_peered":"2026-02-26T03:31:48.405807+0000","last_clean":"2026-02-26T03:31:48.405807+0000","last_became_active":"2026-02-26T03:26:36.662567+0000","last_became_peered":"2026-02-26T03:26:36.662567+0000","last_unstale":"2026-02-26T03:31:48.405807+0000","last_undegraded":"2026-02-26T03:31:48.405807+0000","last_fullsized":"2026-02-26T03:31:48.405807+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.669601+0000","last_change":"2026-02-26T03:26:38.647940+0000","last_active":"2026-02-26T03:31:43.669601+0000","last_peered":"2026-02-26T03:31:43.669601+0000","last_clean":"2026-02-26T03:31:43.669601+0000","last_became_active":"2026-02-26T03:26:36.672454+0000","last_became_peered":"2026-02-26T03:26:36.672454+0000","last_unstale":"2026-02-26T03:31:43.669601+0000","last_undegraded":"2026-02-26T03:31:43.669601+0000","last_fullsized":"2026-02-26T03:31:43.669601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668975+0000","last_change":"2026-02-26T03:26:38.647886+0000","last_active":"2026-02-26T03:31:43.668975+0000","last_peered":"2026-02-26T03:31:43.668975+0000","last_clean":"2026-02-26T03:31:43.668975+0000","last_became_active":"2026-02-26T03:26:36.667431+0000","last_became_peered":"2026-02-26T03:26:36.667431+0000","last_unstale":"2026-02-26T03:31:43.668975+0000","last_undegraded":"2026-02-26T03:31:43.668975+0000","last_fullsized":"2026-02-26T03:31:43.668975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":846,"reported_epoch":292,"state":"active+clean","last_fresh":"2026-02-26T03:31:43.668583+0000","last_change":"2026-02-26T03:26:38.647488+0000","last_active":"2026-02-26T03:31:43.668583+0000","last_peered":"2026-02-26T03:31:43.668583+0000","last_clean":"2026-02-26T03:31:43.668583+0000","last_became_active":"2026-02-26T03:26:36.663643+0000","last_became_peered":"2026-02-26T03:26:36.663643+0000","last_unstale":"2026-02-26T03:31:43.668583+0000","last_undegraded":"2026-02-26T03:31:43.668583+0000","last_fullsized":"2026-02-26T03:31:43.668583+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":73,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":213674622976,"available":213658849280,"internally_reserved":0,"allocated":15773696,"data_stored":15773696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160243818496,"internally_reserved":0,"allocated":12148736,"data_stored":12148736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":4180,"kb_used_data":4180,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162476,"statfs":{"total":53418655744,"available":53414375424,"internally_reserved":0,"allocated":4280320,"data_stored":4280320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3540,"kb_used_data":3540,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163116,"statfs":{"total":53418655744,"available":53415030784,"internally_reserved":0,"allocated":3624960,"data_stored":3624960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673274,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":4148,"kb_used_data":4148,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162508,"statfs":{"total":53418655744,"available":53414408192,"internally_reserved":0,"allocated":4247552,"data_stored":4247552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3492,"kb_used_data":3492,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163164,"statfs":{"total":53418655744,"available":53415079936,"internally_reserved":0,"allocated":3575808,"data_stored":3575808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53414543360,"internally_reserved":0,"allocated":4112384,"data_stored":4112384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53415014400,"internally_reserved":0,"allocated":3641344,"data_stored":3641344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":0,"total":53418655744,"available":53415030784,"internally_reserved":0,"allocated":3624960,"data_stored":3624960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":1,"total":53418655744,"available":53414543360,"internally_reserved":0,"allocated":4112384,"data_stored":4112384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":2,"total":53418655744,"available":53415014400,"internally_reserved":0,"allocated":3641344,"data_stored":3641344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":3,"total":53418655744,"available":53414260736,"internally_reserved":0,"allocated":4395008,"data_stored":4395008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:31:49.633 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-26T03:31:49.709 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:49.704+0000 7f8b0bd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:49.712 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:49.709+0000 7f8b0bd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:49.825 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:31:49.825 INFO:teuthology.orchestra.run.trial012.stdout:{"epoch":295,"fsid":"dc070458-d616-4479-8618-4ead68863657","created":"2026-02-26T03:26:25.317244+0000","modified":"2026-02-26T03:31:48.375066+0000","last_up_change":"2026-02-26T03:26:34.618095+0000","last_in_change":"2026-02-26T03:26:27.184265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":75,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-26T03:26:34.898551+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2026-02-26T03:26:35.564040+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":73,"pool_name":"黅","create_time":"2026-02-26T03:31:35.473628+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"289","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fb2e3d6-0416-4aa3-ac72-b5bc69d2ac65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":292,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6804","nonce":3513533170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6803","nonce":3513533170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6807","nonce":3513533170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6800","nonce":3513533170}]},"public_addr":"10.20.193.12:6804/3513533170","cluster_addr":"10.20.193.12:6803/3513533170","heartbeat_back_addr":"10.20.193.12:6807/3513533170","heartbeat_front_addr":"10.20.193.12:6800/3513533170","state":["exists","up"]},{"osd":1,"uuid":"91f65545-a261-4939-87a5-33cf8e3fa19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":292,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6801","nonce":2473255298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6802","nonce":2473255298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6806","nonce":2473255298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.12:6805","nonce":2473255298}]},"public_addr":"10.20.193.12:6801/2473255298","cluster_addr":"10.20.193.12:6802/2473255298","heartbeat_back_addr":"10.20.193.12:6806/2473255298","heartbeat_front_addr":"10.20.193.12:6805/2473255298","state":["exists","up"]},{"osd":2,"uuid":"e932d28c-883b-4fd0-a0a8-57d9a0f55eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":292,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6804","nonce":256615363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6803","nonce":256615363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6802","nonce":256615363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6805","nonce":256615363}]},"public_addr":"10.20.193.68:6804/256615363","cluster_addr":"10.20.193.68:6803/256615363","heartbeat_back_addr":"10.20.193.68:6802/256615363","heartbeat_front_addr":"10.20.193.68:6805/256615363","state":["exists","up"]},{"osd":3,"uuid":"7adc0e07-7937-4690-bb26-ec324ee9dad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":292,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6810","nonce":1576658665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6809","nonce":1576658665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6808","nonce":1576658665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6807","nonce":1576658665}]},"public_addr":"10.20.193.68:6810/1576658665","cluster_addr":"10.20.193.68:6809/1576658665","heartbeat_back_addr":"10.20.193.68:6808/1576658665","heartbeat_front_addr":"10.20.193.68:6807/1576658665","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]},{"pool":28,"snaps":[{"begin":1,"length":1}]},{"pool":33,"snaps":[{"begin":1,"length":1}]},{"pool":34,"snaps":[{"begin":1,"length":1}]},{"pool":35,"snaps":[{"begin":1,"length":1}]},{"pool":36,"snaps":[{"begin":1,"length":1}]},{"pool":72,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:31:50.836 INFO:tasks.ceph:Scrubbing osd.0 2026-02-26T03:31:50.836 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-26T03:31:50.917 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:50.910+0000 7f70ff593640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:50.920 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:50.917+0000 7f70ff593640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:50.937 INFO:teuthology.orchestra.run.trial012.stdout:{ 2026-02-26T03:31:50.938 INFO:teuthology.orchestra.run.trial012.stdout: "success": "" 2026-02-26T03:31:50.938 INFO:teuthology.orchestra.run.trial012.stdout:} 2026-02-26T03:31:50.945 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-26T03:31:51.022 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.017+0000 7f2b0001a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.025 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.022+0000 7f2b0001a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.137 INFO:teuthology.orchestra.run.trial012.stderr:instructed osd(s) 0 to deep-scrub 2026-02-26T03:31:51.152 INFO:tasks.ceph:Scrubbing osd.1 2026-02-26T03:31:51.152 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-26T03:31:51.231 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.224+0000 7f514f25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.233 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.231+0000 7f514f25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.243 INFO:teuthology.orchestra.run.trial012.stdout:{ 2026-02-26T03:31:51.243 INFO:teuthology.orchestra.run.trial012.stdout: "success": "" 2026-02-26T03:31:51.244 INFO:teuthology.orchestra.run.trial012.stdout:} 2026-02-26T03:31:51.251 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-26T03:31:51.327 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.321+0000 7ff301db8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.329 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.326+0000 7ff301db8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.439 INFO:teuthology.orchestra.run.trial012.stderr:instructed osd(s) 1 to deep-scrub 2026-02-26T03:31:51.451 INFO:tasks.ceph:Scrubbing osd.2 2026-02-26T03:31:51.451 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-26T03:31:51.530 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.523+0000 7f4ef93dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.532 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.530+0000 7f4ef93dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.542 INFO:teuthology.orchestra.run.trial012.stdout:{ 2026-02-26T03:31:51.543 INFO:teuthology.orchestra.run.trial012.stdout: "success": "" 2026-02-26T03:31:51.543 INFO:teuthology.orchestra.run.trial012.stdout:} 2026-02-26T03:31:51.549 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-26T03:31:51.626 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.620+0000 7f5146046640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.629 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.626+0000 7f5146046640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.739 INFO:teuthology.orchestra.run.trial012.stderr:instructed osd(s) 2 to deep-scrub 2026-02-26T03:31:51.750 INFO:tasks.ceph:Scrubbing osd.3 2026-02-26T03:31:51.750 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-26T03:31:51.828 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.821+0000 7f11eeecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.831 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.828+0000 7f11eeecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.843 INFO:teuthology.orchestra.run.trial012.stdout:{ 2026-02-26T03:31:51.843 INFO:teuthology.orchestra.run.trial012.stdout: "success": "" 2026-02-26T03:31:51.843 INFO:teuthology.orchestra.run.trial012.stdout:} 2026-02-26T03:31:51.850 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-26T03:31:51.926 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.921+0000 7f58fa65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:51.929 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:51.926+0000 7f58fa65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:52.040 INFO:teuthology.orchestra.run.trial012.stderr:instructed osd(s) 3 to deep-scrub 2026-02-26T03:31:52.051 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:31:52.128 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:52.123+0000 7f884bc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:52.130 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:31:52.128+0000 7f884bc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:31:52.241 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:31:52.242 INFO:teuthology.orchestra.run.trial012.stderr:dumped all 2026-02-26T03:31:52.250 INFO:teuthology.orchestra.run.trial012.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2026-02-26T03:31:51.597605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":15156,"kb_used_data":15156,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208651468,"statfs":{"total":213674622976,"available":213659103232,"internally_reserved":0,"allocated":15519744,"data_stored":15519744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.225666"},"pg_stats":[{"pgid":"73.f","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.404166+0000","last_change":"2026-02-26T03:31:37.534903+0000","last_active":"2026-02-26T03:31:48.404166+0000","last_peered":"2026-02-26T03:31:48.404166+0000","last_clean":"2026-02-26T03:31:48.404166+0000","last_became_active":"2026-02-26T03:31:37.512557+0000","last_became_peered":"2026-02-26T03:31:37.512557+0000","last_unstale":"2026-02-26T03:31:48.404166+0000","last_undegraded":"2026-02-26T03:31:48.404166+0000","last_fullsized":"2026-02-26T03:31:48.404166+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.e","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.403682+0000","last_change":"2026-02-26T03:31:37.496100+0000","last_active":"2026-02-26T03:31:48.403682+0000","last_peered":"2026-02-26T03:31:48.403682+0000","last_clean":"2026-02-26T03:31:48.403682+0000","last_became_active":"2026-02-26T03:31:37.489234+0000","last_became_peered":"2026-02-26T03:31:37.489234+0000","last_unstale":"2026-02-26T03:31:48.403682+0000","last_undegraded":"2026-02-26T03:31:48.403682+0000","last_fullsized":"2026-02-26T03:31:48.403682+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.d","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.391759+0000","last_change":"2026-02-26T03:31:37.512242+0000","last_active":"2026-02-26T03:31:48.391759+0000","last_peered":"2026-02-26T03:31:48.391759+0000","last_clean":"2026-02-26T03:31:48.391759+0000","last_became_active":"2026-02-26T03:31:37.505462+0000","last_became_peered":"2026-02-26T03:31:37.505462+0000","last_unstale":"2026-02-26T03:31:48.391759+0000","last_undegraded":"2026-02-26T03:31:48.391759+0000","last_fullsized":"2026-02-26T03:31:48.391759+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.c","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.401427+0000","last_change":"2026-02-26T03:31:37.508762+0000","last_active":"2026-02-26T03:31:48.401427+0000","last_peered":"2026-02-26T03:31:48.401427+0000","last_clean":"2026-02-26T03:31:48.401427+0000","last_became_active":"2026-02-26T03:31:37.495191+0000","last_became_peered":"2026-02-26T03:31:37.495191+0000","last_unstale":"2026-02-26T03:31:48.401427+0000","last_undegraded":"2026-02-26T03:31:48.401427+0000","last_fullsized":"2026-02-26T03:31:48.401427+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.b","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.397812+0000","last_change":"2026-02-26T03:31:37.503079+0000","last_active":"2026-02-26T03:31:48.397812+0000","last_peered":"2026-02-26T03:31:48.397812+0000","last_clean":"2026-02-26T03:31:48.397812+0000","last_became_active":"2026-02-26T03:31:37.497201+0000","last_became_peered":"2026-02-26T03:31:37.497201+0000","last_unstale":"2026-02-26T03:31:48.397812+0000","last_undegraded":"2026-02-26T03:31:48.397812+0000","last_fullsized":"2026-02-26T03:31:48.397812+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.a","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.396154+0000","last_change":"2026-02-26T03:31:37.527767+0000","last_active":"2026-02-26T03:31:48.396154+0000","last_peered":"2026-02-26T03:31:48.396154+0000","last_clean":"2026-02-26T03:31:48.396154+0000","last_became_active":"2026-02-26T03:31:37.506862+0000","last_became_peered":"2026-02-26T03:31:37.506862+0000","last_unstale":"2026-02-26T03:31:48.396154+0000","last_undegraded":"2026-02-26T03:31:48.396154+0000","last_fullsized":"2026-02-26T03:31:48.396154+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.9","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.403208+0000","last_change":"2026-02-26T03:31:37.496374+0000","last_active":"2026-02-26T03:31:48.403208+0000","last_peered":"2026-02-26T03:31:48.403208+0000","last_clean":"2026-02-26T03:31:48.403208+0000","last_became_active":"2026-02-26T03:31:37.489901+0000","last_became_peered":"2026-02-26T03:31:37.489901+0000","last_unstale":"2026-02-26T03:31:48.403208+0000","last_undegraded":"2026-02-26T03:31:48.403208+0000","last_fullsized":"2026-02-26T03:31:48.403208+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.8","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.396657+0000","last_change":"2026-02-26T03:31:37.493086+0000","last_active":"2026-02-26T03:31:48.396657+0000","last_peered":"2026-02-26T03:31:48.396657+0000","last_clean":"2026-02-26T03:31:48.396657+0000","last_became_active":"2026-02-26T03:31:37.484153+0000","last_became_peered":"2026-02-26T03:31:37.484153+0000","last_unstale":"2026-02-26T03:31:48.396657+0000","last_undegraded":"2026-02-26T03:31:48.396657+0000","last_fullsized":"2026-02-26T03:31:48.396657+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.7","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.399926+0000","last_change":"2026-02-26T03:31:37.553536+0000","last_active":"2026-02-26T03:31:48.399926+0000","last_peered":"2026-02-26T03:31:48.399926+0000","last_clean":"2026-02-26T03:31:48.399926+0000","last_became_active":"2026-02-26T03:31:37.522635+0000","last_became_peered":"2026-02-26T03:31:37.522635+0000","last_unstale":"2026-02-26T03:31:48.399926+0000","last_undegraded":"2026-02-26T03:31:48.399926+0000","last_fullsized":"2026-02-26T03:31:48.399926+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.6","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.399781+0000","last_change":"2026-02-26T03:31:37.515418+0000","last_active":"2026-02-26T03:31:48.399781+0000","last_peered":"2026-02-26T03:31:48.399781+0000","last_clean":"2026-02-26T03:31:48.399781+0000","last_became_active":"2026-02-26T03:31:37.492882+0000","last_became_peered":"2026-02-26T03:31:37.492882+0000","last_unstale":"2026-02-26T03:31:48.399781+0000","last_undegraded":"2026-02-26T03:31:48.399781+0000","last_fullsized":"2026-02-26T03:31:48.399781+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.5","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.399235+0000","last_change":"2026-02-26T03:31:37.551105+0000","last_active":"2026-02-26T03:31:48.399235+0000","last_peered":"2026-02-26T03:31:48.399235+0000","last_clean":"2026-02-26T03:31:48.399235+0000","last_became_active":"2026-02-26T03:31:37.544290+0000","last_became_peered":"2026-02-26T03:31:37.544290+0000","last_unstale":"2026-02-26T03:31:48.399235+0000","last_undegraded":"2026-02-26T03:31:48.399235+0000","last_fullsized":"2026-02-26T03:31:48.399235+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.4","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.399764+0000","last_change":"2026-02-26T03:31:37.510764+0000","last_active":"2026-02-26T03:31:48.399764+0000","last_peered":"2026-02-26T03:31:48.399764+0000","last_clean":"2026-02-26T03:31:48.399764+0000","last_became_active":"2026-02-26T03:31:37.506074+0000","last_became_peered":"2026-02-26T03:31:37.506074+0000","last_unstale":"2026-02-26T03:31:48.399764+0000","last_undegraded":"2026-02-26T03:31:48.399764+0000","last_fullsized":"2026-02-26T03:31:48.399764+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.3","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.400807+0000","last_change":"2026-02-26T03:31:37.561329+0000","last_active":"2026-02-26T03:31:48.400807+0000","last_peered":"2026-02-26T03:31:48.400807+0000","last_clean":"2026-02-26T03:31:48.400807+0000","last_became_active":"2026-02-26T03:31:37.554277+0000","last_became_peered":"2026-02-26T03:31:37.554277+0000","last_unstale":"2026-02-26T03:31:48.400807+0000","last_undegraded":"2026-02-26T03:31:48.400807+0000","last_fullsized":"2026-02-26T03:31:48.400807+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.2","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.404442+0000","last_change":"2026-02-26T03:31:37.524339+0000","last_active":"2026-02-26T03:31:48.404442+0000","last_peered":"2026-02-26T03:31:48.404442+0000","last_clean":"2026-02-26T03:31:48.404442+0000","last_became_active":"2026-02-26T03:31:37.518931+0000","last_became_peered":"2026-02-26T03:31:37.518931+0000","last_unstale":"2026-02-26T03:31:48.404442+0000","last_undegraded":"2026-02-26T03:31:48.404442+0000","last_fullsized":"2026-02-26T03:31:48.404442+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.1","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.395829+0000","last_change":"2026-02-26T03:31:37.576337+0000","last_active":"2026-02-26T03:31:48.395829+0000","last_peered":"2026-02-26T03:31:48.395829+0000","last_clean":"2026-02-26T03:31:48.395829+0000","last_became_active":"2026-02-26T03:31:37.556206+0000","last_became_peered":"2026-02-26T03:31:37.556206+0000","last_unstale":"2026-02-26T03:31:48.395829+0000","last_undegraded":"2026-02-26T03:31:48.395829+0000","last_fullsized":"2026-02-26T03:31:48.395829+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.0","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.396990+0000","last_change":"2026-02-26T03:31:37.507650+0000","last_active":"2026-02-26T03:31:48.396990+0000","last_peered":"2026-02-26T03:31:48.396990+0000","last_clean":"2026-02-26T03:31:48.396990+0000","last_became_active":"2026-02-26T03:31:37.493062+0000","last_became_peered":"2026-02-26T03:31:37.493062+0000","last_unstale":"2026-02-26T03:31:48.396990+0000","last_undegraded":"2026-02-26T03:31:48.396990+0000","last_fullsized":"2026-02-26T03:31:48.396990+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:36.435649+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.398176+0000","last_change":"2026-02-26T03:26:38.647558+0000","last_active":"2026-02-26T03:31:48.398176+0000","last_peered":"2026-02-26T03:31:48.398176+0000","last_clean":"2026-02-26T03:31:48.398176+0000","last_became_active":"2026-02-26T03:26:36.658652+0000","last_became_peered":"2026-02-26T03:26:36.658652+0000","last_unstale":"2026-02-26T03:31:48.398176+0000","last_undegraded":"2026-02-26T03:31:48.398176+0000","last_fullsized":"2026-02-26T03:31:48.398176+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.401297+0000","last_change":"2026-02-26T03:26:38.648362+0000","last_active":"2026-02-26T03:31:48.401297+0000","last_peered":"2026-02-26T03:31:48.401297+0000","last_clean":"2026-02-26T03:31:48.401297+0000","last_became_active":"2026-02-26T03:26:36.669960+0000","last_became_peered":"2026-02-26T03:26:36.669960+0000","last_unstale":"2026-02-26T03:31:48.401297+0000","last_undegraded":"2026-02-26T03:31:48.401297+0000","last_fullsized":"2026-02-26T03:31:48.401297+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":857,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.396913+0000","last_change":"2026-02-26T03:26:38.646292+0000","last_active":"2026-02-26T03:31:48.396913+0000","last_peered":"2026-02-26T03:31:48.396913+0000","last_clean":"2026-02-26T03:31:48.396913+0000","last_became_active":"2026-02-26T03:26:36.660207+0000","last_became_peered":"2026-02-26T03:26:36.660207+0000","last_unstale":"2026-02-26T03:31:48.396913+0000","last_undegraded":"2026-02-26T03:31:48.396913+0000","last_fullsized":"2026-02-26T03:31:48.396913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":856,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.398292+0000","last_change":"2026-02-26T03:26:38.645479+0000","last_active":"2026-02-26T03:31:48.398292+0000","last_peered":"2026-02-26T03:31:48.398292+0000","last_clean":"2026-02-26T03:31:48.398292+0000","last_became_active":"2026-02-26T03:26:36.666405+0000","last_became_peered":"2026-02-26T03:26:36.666405+0000","last_unstale":"2026-02-26T03:31:48.398292+0000","last_undegraded":"2026-02-26T03:31:48.398292+0000","last_fullsized":"2026-02-26T03:31:48.398292+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":921,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.399937+0000","last_change":"2026-02-26T03:26:36.680586+0000","last_active":"2026-02-26T03:31:48.399937+0000","last_peered":"2026-02-26T03:31:48.399937+0000","last_clean":"2026-02-26T03:31:48.399937+0000","last_became_active":"2026-02-26T03:26:36.667237+0000","last_became_peered":"2026-02-26T03:26:36.667237+0000","last_unstale":"2026-02-26T03:31:48.399937+0000","last_undegraded":"2026-02-26T03:31:48.399937+0000","last_fullsized":"2026-02-26T03:31:48.399937+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.405807+0000","last_change":"2026-02-26T03:26:38.646875+0000","last_active":"2026-02-26T03:31:48.405807+0000","last_peered":"2026-02-26T03:31:48.405807+0000","last_clean":"2026-02-26T03:31:48.405807+0000","last_became_active":"2026-02-26T03:26:36.662567+0000","last_became_peered":"2026-02-26T03:26:36.662567+0000","last_unstale":"2026-02-26T03:31:48.405807+0000","last_undegraded":"2026-02-26T03:31:48.405807+0000","last_fullsized":"2026-02-26T03:31:48.405807+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.400454+0000","last_change":"2026-02-26T03:26:38.647940+0000","last_active":"2026-02-26T03:31:48.400454+0000","last_peered":"2026-02-26T03:31:48.400454+0000","last_clean":"2026-02-26T03:31:48.400454+0000","last_became_active":"2026-02-26T03:26:36.672454+0000","last_became_peered":"2026-02-26T03:26:36.672454+0000","last_unstale":"2026-02-26T03:31:48.400454+0000","last_undegraded":"2026-02-26T03:31:48.400454+0000","last_fullsized":"2026-02-26T03:31:48.400454+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.402943+0000","last_change":"2026-02-26T03:26:38.647886+0000","last_active":"2026-02-26T03:31:48.402943+0000","last_peered":"2026-02-26T03:31:48.402943+0000","last_clean":"2026-02-26T03:31:48.402943+0000","last_became_active":"2026-02-26T03:26:36.667431+0000","last_became_peered":"2026-02-26T03:26:36.667431+0000","last_unstale":"2026-02-26T03:31:48.402943+0000","last_undegraded":"2026-02-26T03:31:48.402943+0000","last_fullsized":"2026-02-26T03:31:48.402943+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:31:48.398388+0000","last_change":"2026-02-26T03:26:38.647488+0000","last_active":"2026-02-26T03:31:48.398388+0000","last_peered":"2026-02-26T03:31:48.398388+0000","last_clean":"2026-02-26T03:31:48.398388+0000","last_became_active":"2026-02-26T03:26:36.663643+0000","last_became_peered":"2026-02-26T03:26:36.663643+0000","last_unstale":"2026-02-26T03:31:48.398388+0000","last_undegraded":"2026-02-26T03:31:48.398388+0000","last_fullsized":"2026-02-26T03:31:48.398388+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:26:35.622204+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":73,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":213674622976,"available":213659238400,"internally_reserved":0,"allocated":15384576,"data_stored":15384576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53414445056,"internally_reserved":0,"allocated":4210688,"data_stored":4210688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160244088832,"internally_reserved":0,"allocated":11878400,"data_stored":11878400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":4112,"kb_used_data":4112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162544,"statfs":{"total":53418655744,"available":53414445056,"internally_reserved":0,"allocated":4210688,"data_stored":4210688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3472,"kb_used_data":3472,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163184,"statfs":{"total":53418655744,"available":53415100416,"internally_reserved":0,"allocated":3555328,"data_stored":3555328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673274,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":4148,"kb_used_data":4148,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162508,"statfs":{"total":53418655744,"available":53414408192,"internally_reserved":0,"allocated":4247552,"data_stored":4247552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3424,"kb_used_data":3424,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163232,"statfs":{"total":53418655744,"available":53415149568,"internally_reserved":0,"allocated":3506176,"data_stored":3506176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53414543360,"internally_reserved":0,"allocated":4112384,"data_stored":4112384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53415100416,"internally_reserved":0,"allocated":3555328,"data_stored":3555328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53414445056,"internally_reserved":0,"allocated":4210688,"data_stored":4210688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53418655744,"available":53414445056,"internally_reserved":0,"allocated":4210688,"data_stored":4210688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":0,"total":53418655744,"available":53415149568,"internally_reserved":0,"allocated":3506176,"data_stored":3506176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":1,"total":53418655744,"available":53414543360,"internally_reserved":0,"allocated":4112384,"data_stored":4112384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":2,"total":53418655744,"available":53415100416,"internally_reserved":0,"allocated":3555328,"data_stored":3555328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":3,"total":53418655744,"available":53414445056,"internally_reserved":0,"allocated":4210688,"data_stored":4210688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:31:52.257 INFO:tasks.ceph:pgid 73.f last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.257 INFO:tasks.ceph:pgid 73.e last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.257 INFO:tasks.ceph:pgid 73.d last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.258 INFO:tasks.ceph:pgid 73.c last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.258 INFO:tasks.ceph:pgid 73.b last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.258 INFO:tasks.ceph:pgid 73.a last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.258 INFO:tasks.ceph:pgid 73.9 last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.258 INFO:tasks.ceph:pgid 73.8 last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.258 INFO:tasks.ceph:pgid 73.7 last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.258 INFO:tasks.ceph:pgid 73.6 last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.259 INFO:tasks.ceph:pgid 73.5 last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.259 INFO:tasks.ceph:pgid 73.4 last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.259 INFO:tasks.ceph:pgid 73.3 last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.259 INFO:tasks.ceph:pgid 73.2 last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.259 INFO:tasks.ceph:pgid 73.1 last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.259 INFO:tasks.ceph:pgid 73.0 last_scrub_stamp 2026-02-26T03:31:36.435649+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=36, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.259 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-26T03:26:35.622204+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.260 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-26T03:26:35.622204+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.260 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2026-02-26T03:26:35.622204+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.260 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-26T03:26:35.622204+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.260 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-26T03:26:35.622204+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.260 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-26T03:26:35.622204+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.260 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-26T03:26:35.622204+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.260 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-26T03:26:35.622204+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.261 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-26T03:26:35.622204+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=31, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:31:52.261 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-26T03:32:12.262 DEBUG:teuthology.orchestra.run.trial012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:32:12.341 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:32:12.337+0000 7f993035c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:32:12.344 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:32:12.342+0000 7f993035c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:32:12.454 INFO:teuthology.orchestra.run.trial012.stdout: 2026-02-26T03:32:12.455 INFO:teuthology.orchestra.run.trial012.stderr:dumped all 2026-02-26T03:32:12.464 INFO:teuthology.orchestra.run.trial012.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2026-02-26T03:32:11.600775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":15056,"kb_used_data":15056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208651568,"statfs":{"total":213674622976,"available":213659205632,"internally_reserved":0,"allocated":15417344,"data_stored":15417344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001904"},"pg_stats":[{"pgid":"73.f","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.742128+0000","last_change":"2026-02-26T03:31:51.757511+0000","last_active":"2026-02-26T03:31:51.742128+0000","last_peered":"2026-02-26T03:31:51.742128+0000","last_clean":"2026-02-26T03:31:51.742128+0000","last_became_active":"2026-02-26T03:31:37.512557+0000","last_became_peered":"2026-02-26T03:31:37.512557+0000","last_unstale":"2026-02-26T03:31:51.742128+0000","last_undegraded":"2026-02-26T03:31:51.742128+0000","last_fullsized":"2026-02-26T03:31:51.742128+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.757435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.757435+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.e","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.441091+0000","last_change":"2026-02-26T03:31:51.450141+0000","last_active":"2026-02-26T03:31:51.441091+0000","last_peered":"2026-02-26T03:31:51.441091+0000","last_clean":"2026-02-26T03:31:51.441091+0000","last_became_active":"2026-02-26T03:31:37.489234+0000","last_became_peered":"2026-02-26T03:31:37.489234+0000","last_unstale":"2026-02-26T03:31:51.441091+0000","last_undegraded":"2026-02-26T03:31:51.441091+0000","last_fullsized":"2026-02-26T03:31:51.441091+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.450033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.450033+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.d","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.740978+0000","last_change":"2026-02-26T03:31:51.756864+0000","last_active":"2026-02-26T03:31:51.740978+0000","last_peered":"2026-02-26T03:31:51.740978+0000","last_clean":"2026-02-26T03:31:51.740978+0000","last_became_active":"2026-02-26T03:31:37.505462+0000","last_became_peered":"2026-02-26T03:31:37.505462+0000","last_unstale":"2026-02-26T03:31:51.740978+0000","last_undegraded":"2026-02-26T03:31:51.740978+0000","last_fullsized":"2026-02-26T03:31:51.740978+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.756816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.756816+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.c","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.741388+0000","last_change":"2026-02-26T03:31:51.755979+0000","last_active":"2026-02-26T03:31:51.741388+0000","last_peered":"2026-02-26T03:31:51.741388+0000","last_clean":"2026-02-26T03:31:51.741388+0000","last_became_active":"2026-02-26T03:31:37.495191+0000","last_became_peered":"2026-02-26T03:31:37.495191+0000","last_unstale":"2026-02-26T03:31:51.741388+0000","last_undegraded":"2026-02-26T03:31:51.741388+0000","last_fullsized":"2026-02-26T03:31:51.741388+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.755927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.755927+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.b","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.739780+0000","last_change":"2026-02-26T03:31:51.749388+0000","last_active":"2026-02-26T03:31:51.739780+0000","last_peered":"2026-02-26T03:31:51.739780+0000","last_clean":"2026-02-26T03:31:51.739780+0000","last_became_active":"2026-02-26T03:31:37.497201+0000","last_became_peered":"2026-02-26T03:31:37.497201+0000","last_unstale":"2026-02-26T03:31:51.739780+0000","last_undegraded":"2026-02-26T03:31:51.739780+0000","last_fullsized":"2026-02-26T03:31:51.739780+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.749328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.749328+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.a","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:52.043438+0000","last_change":"2026-02-26T03:31:52.064490+0000","last_active":"2026-02-26T03:31:52.043438+0000","last_peered":"2026-02-26T03:31:52.043438+0000","last_clean":"2026-02-26T03:31:52.043438+0000","last_became_active":"2026-02-26T03:31:37.506862+0000","last_became_peered":"2026-02-26T03:31:37.506862+0000","last_unstale":"2026-02-26T03:31:52.043438+0000","last_undegraded":"2026-02-26T03:31:52.043438+0000","last_fullsized":"2026-02-26T03:31:52.043438+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:52.064441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:52.064441+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.9","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.441201+0000","last_change":"2026-02-26T03:31:51.453590+0000","last_active":"2026-02-26T03:31:51.441201+0000","last_peered":"2026-02-26T03:31:51.441201+0000","last_clean":"2026-02-26T03:31:51.441201+0000","last_became_active":"2026-02-26T03:31:37.489901+0000","last_became_peered":"2026-02-26T03:31:37.489901+0000","last_unstale":"2026-02-26T03:31:51.441201+0000","last_undegraded":"2026-02-26T03:31:51.441201+0000","last_fullsized":"2026-02-26T03:31:51.441201+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.453539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.453539+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.8","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.739360+0000","last_change":"2026-02-26T03:31:51.758109+0000","last_active":"2026-02-26T03:31:51.739360+0000","last_peered":"2026-02-26T03:31:51.739360+0000","last_clean":"2026-02-26T03:31:51.739360+0000","last_became_active":"2026-02-26T03:31:37.484153+0000","last_became_peered":"2026-02-26T03:31:37.484153+0000","last_unstale":"2026-02-26T03:31:51.739360+0000","last_undegraded":"2026-02-26T03:31:51.739360+0000","last_fullsized":"2026-02-26T03:31:51.739360+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.758060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.758060+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.7","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:52.043245+0000","last_change":"2026-02-26T03:31:52.065548+0000","last_active":"2026-02-26T03:31:52.043245+0000","last_peered":"2026-02-26T03:31:52.043245+0000","last_clean":"2026-02-26T03:31:52.043245+0000","last_became_active":"2026-02-26T03:31:37.522635+0000","last_became_peered":"2026-02-26T03:31:37.522635+0000","last_unstale":"2026-02-26T03:31:52.043245+0000","last_undegraded":"2026-02-26T03:31:52.043245+0000","last_fullsized":"2026-02-26T03:31:52.043245+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:52.065499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:52.065499+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.6","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.739523+0000","last_change":"2026-02-26T03:31:51.752256+0000","last_active":"2026-02-26T03:31:51.739523+0000","last_peered":"2026-02-26T03:31:51.739523+0000","last_clean":"2026-02-26T03:31:51.739523+0000","last_became_active":"2026-02-26T03:31:37.492882+0000","last_became_peered":"2026-02-26T03:31:37.492882+0000","last_unstale":"2026-02-26T03:31:51.739523+0000","last_undegraded":"2026-02-26T03:31:51.739523+0000","last_fullsized":"2026-02-26T03:31:51.739523+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.752178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.752178+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.5","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.739857+0000","last_change":"2026-02-26T03:31:51.752917+0000","last_active":"2026-02-26T03:31:51.739857+0000","last_peered":"2026-02-26T03:31:51.739857+0000","last_clean":"2026-02-26T03:31:51.739857+0000","last_became_active":"2026-02-26T03:31:37.544290+0000","last_became_peered":"2026-02-26T03:31:37.544290+0000","last_unstale":"2026-02-26T03:31:51.739857+0000","last_undegraded":"2026-02-26T03:31:51.739857+0000","last_fullsized":"2026-02-26T03:31:51.739857+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.752867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.752867+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.4","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.139150+0000","last_change":"2026-02-26T03:31:51.147076+0000","last_active":"2026-02-26T03:31:51.139150+0000","last_peered":"2026-02-26T03:31:51.139150+0000","last_clean":"2026-02-26T03:31:51.139150+0000","last_became_active":"2026-02-26T03:31:37.506074+0000","last_became_peered":"2026-02-26T03:31:37.506074+0000","last_unstale":"2026-02-26T03:31:51.139150+0000","last_undegraded":"2026-02-26T03:31:51.139150+0000","last_fullsized":"2026-02-26T03:31:51.139150+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.147013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.147013+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.3","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.138647+0000","last_change":"2026-02-26T03:31:51.146853+0000","last_active":"2026-02-26T03:31:51.138647+0000","last_peered":"2026-02-26T03:31:51.138647+0000","last_clean":"2026-02-26T03:31:51.138647+0000","last_became_active":"2026-02-26T03:31:37.554277+0000","last_became_peered":"2026-02-26T03:31:37.554277+0000","last_unstale":"2026-02-26T03:31:51.138647+0000","last_undegraded":"2026-02-26T03:31:51.138647+0000","last_fullsized":"2026-02-26T03:31:51.138647+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.146762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.146762+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.2","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.440627+0000","last_change":"2026-02-26T03:31:51.451009+0000","last_active":"2026-02-26T03:31:51.440627+0000","last_peered":"2026-02-26T03:31:51.440627+0000","last_clean":"2026-02-26T03:31:51.440627+0000","last_became_active":"2026-02-26T03:31:37.518931+0000","last_became_peered":"2026-02-26T03:31:37.518931+0000","last_unstale":"2026-02-26T03:31:51.440627+0000","last_undegraded":"2026-02-26T03:31:51.440627+0000","last_fullsized":"2026-02-26T03:31:51.440627+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.450951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.450951+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.1","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:52.041716+0000","last_change":"2026-02-26T03:31:52.065929+0000","last_active":"2026-02-26T03:31:52.041716+0000","last_peered":"2026-02-26T03:31:52.041716+0000","last_clean":"2026-02-26T03:31:52.041716+0000","last_became_active":"2026-02-26T03:31:37.556206+0000","last_became_peered":"2026-02-26T03:31:37.556206+0000","last_unstale":"2026-02-26T03:31:52.041716+0000","last_undegraded":"2026-02-26T03:31:52.041716+0000","last_fullsized":"2026-02-26T03:31:52.041716+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:52.065880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:52.065880+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.0","version":"0'0","reported_seq":34,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.137912+0000","last_change":"2026-02-26T03:31:51.146191+0000","last_active":"2026-02-26T03:31:51.137912+0000","last_peered":"2026-02-26T03:31:51.137912+0000","last_clean":"2026-02-26T03:31:51.137912+0000","last_became_active":"2026-02-26T03:31:37.493062+0000","last_became_peered":"2026-02-26T03:31:37.493062+0000","last_unstale":"2026-02-26T03:31:51.137912+0000","last_undegraded":"2026-02-26T03:31:51.137912+0000","last_fullsized":"2026-02-26T03:31:51.137912+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.146130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.146130+0000","last_clean_scrub_stamp":"2026-02-26T03:31:36.435649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":860,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:52.038686+0000","last_change":"2026-02-26T03:31:52.063422+0000","last_active":"2026-02-26T03:31:52.038686+0000","last_peered":"2026-02-26T03:31:52.038686+0000","last_clean":"2026-02-26T03:31:52.038686+0000","last_became_active":"2026-02-26T03:26:36.658652+0000","last_became_peered":"2026-02-26T03:26:36.658652+0000","last_unstale":"2026-02-26T03:31:52.038686+0000","last_undegraded":"2026-02-26T03:31:52.038686+0000","last_fullsized":"2026-02-26T03:31:52.038686+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:52.063360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:52.063360+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":860,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:52.039227+0000","last_change":"2026-02-26T03:31:52.052109+0000","last_active":"2026-02-26T03:31:52.039227+0000","last_peered":"2026-02-26T03:31:52.039227+0000","last_clean":"2026-02-26T03:31:52.039227+0000","last_became_active":"2026-02-26T03:26:36.669960+0000","last_became_peered":"2026-02-26T03:26:36.669960+0000","last_unstale":"2026-02-26T03:31:52.039227+0000","last_undegraded":"2026-02-26T03:31:52.039227+0000","last_fullsized":"2026-02-26T03:31:52.039227+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:52.052058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:52.052058+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":862,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.737569+0000","last_change":"2026-02-26T03:31:51.761682+0000","last_active":"2026-02-26T03:31:51.737569+0000","last_peered":"2026-02-26T03:31:51.737569+0000","last_clean":"2026-02-26T03:31:51.737569+0000","last_became_active":"2026-02-26T03:26:36.660207+0000","last_became_peered":"2026-02-26T03:26:36.660207+0000","last_unstale":"2026-02-26T03:31:51.737569+0000","last_undegraded":"2026-02-26T03:31:51.737569+0000","last_fullsized":"2026-02-26T03:31:51.737569+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2026-02-26T03:31:51.761630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.761630+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":861,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.439546+0000","last_change":"2026-02-26T03:31:51.457666+0000","last_active":"2026-02-26T03:31:51.439546+0000","last_peered":"2026-02-26T03:31:51.439546+0000","last_clean":"2026-02-26T03:31:51.439546+0000","last_became_active":"2026-02-26T03:26:36.666405+0000","last_became_peered":"2026-02-26T03:26:36.666405+0000","last_unstale":"2026-02-26T03:31:51.439546+0000","last_undegraded":"2026-02-26T03:31:51.439546+0000","last_fullsized":"2026-02-26T03:31:51.439546+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"13'1","last_scrub_stamp":"2026-02-26T03:31:51.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.457608+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":926,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:52.041418+0000","last_change":"2026-02-26T03:31:52.061365+0000","last_active":"2026-02-26T03:31:52.041418+0000","last_peered":"2026-02-26T03:31:52.041418+0000","last_clean":"2026-02-26T03:31:52.041418+0000","last_became_active":"2026-02-26T03:26:36.667237+0000","last_became_peered":"2026-02-26T03:26:36.667237+0000","last_unstale":"2026-02-26T03:31:52.041418+0000","last_undegraded":"2026-02-26T03:31:52.041418+0000","last_fullsized":"2026-02-26T03:31:52.041418+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"13'52","last_scrub_stamp":"2026-02-26T03:31:52.061271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:52.061271+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":860,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.440221+0000","last_change":"2026-02-26T03:31:51.451396+0000","last_active":"2026-02-26T03:31:51.440221+0000","last_peered":"2026-02-26T03:31:51.440221+0000","last_clean":"2026-02-26T03:31:51.440221+0000","last_became_active":"2026-02-26T03:26:36.662567+0000","last_became_peered":"2026-02-26T03:26:36.662567+0000","last_unstale":"2026-02-26T03:31:51.440221+0000","last_undegraded":"2026-02-26T03:31:51.440221+0000","last_fullsized":"2026-02-26T03:31:51.440221+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.451316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.451316+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":860,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:51.738552+0000","last_change":"2026-02-26T03:31:51.751383+0000","last_active":"2026-02-26T03:31:51.738552+0000","last_peered":"2026-02-26T03:31:51.738552+0000","last_clean":"2026-02-26T03:31:51.738552+0000","last_became_active":"2026-02-26T03:26:36.672454+0000","last_became_peered":"2026-02-26T03:26:36.672454+0000","last_unstale":"2026-02-26T03:31:51.738552+0000","last_undegraded":"2026-02-26T03:31:51.738552+0000","last_fullsized":"2026-02-26T03:31:51.738552+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:51.751328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:51.751328+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":860,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:52.039953+0000","last_change":"2026-02-26T03:31:52.051464+0000","last_active":"2026-02-26T03:31:52.039953+0000","last_peered":"2026-02-26T03:31:52.039953+0000","last_clean":"2026-02-26T03:31:52.039953+0000","last_became_active":"2026-02-26T03:26:36.667431+0000","last_became_peered":"2026-02-26T03:26:36.667431+0000","last_unstale":"2026-02-26T03:31:52.039953+0000","last_undegraded":"2026-02-26T03:31:52.039953+0000","last_fullsized":"2026-02-26T03:31:52.039953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:52.051377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:52.051377+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":860,"reported_epoch":296,"state":"active+clean","last_fresh":"2026-02-26T03:31:52.041028+0000","last_change":"2026-02-26T03:31:52.065126+0000","last_active":"2026-02-26T03:31:52.041028+0000","last_peered":"2026-02-26T03:31:52.041028+0000","last_clean":"2026-02-26T03:31:52.041028+0000","last_became_active":"2026-02-26T03:26:36.663643+0000","last_became_peered":"2026-02-26T03:26:36.663643+0000","last_unstale":"2026-02-26T03:31:52.041028+0000","last_undegraded":"2026-02-26T03:31:52.041028+0000","last_fullsized":"2026-02-26T03:31:52.041028+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:31:52.065077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:31:52.065077+0000","last_clean_scrub_stamp":"2026-02-26T03:26:35.622204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":73,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":213674622976,"available":213659205632,"internally_reserved":0,"allocated":15417344,"data_stored":15417344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53414436864,"internally_reserved":0,"allocated":4218880,"data_stored":4218880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160244064256,"internally_reserved":0,"allocated":11902976,"data_stored":11902976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":4120,"kb_used_data":4120,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162536,"statfs":{"total":53418655744,"available":53414436864,"internally_reserved":0,"allocated":4218880,"data_stored":4218880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3480,"kb_used_data":3480,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163176,"statfs":{"total":53418655744,"available":53415092224,"internally_reserved":0,"allocated":3563520,"data_stored":3563520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673294,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":4024,"kb_used_data":4024,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52162632,"statfs":{"total":53418655744,"available":53414535168,"internally_reserved":0,"allocated":4120576,"data_stored":4120576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":3432,"kb_used_data":3432,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163224,"statfs":{"total":53418655744,"available":53415141376,"internally_reserved":0,"allocated":3514368,"data_stored":3514368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53414535168,"internally_reserved":0,"allocated":4120576,"data_stored":4120576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53415092224,"internally_reserved":0,"allocated":3563520,"data_stored":3563520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53414436864,"internally_reserved":0,"allocated":4218880,"data_stored":4218880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53418655744,"available":53414436864,"internally_reserved":0,"allocated":4218880,"data_stored":4218880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":0,"total":53418655744,"available":53415141376,"internally_reserved":0,"allocated":3514368,"data_stored":3514368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":1,"total":53418655744,"available":53414535168,"internally_reserved":0,"allocated":4120576,"data_stored":4120576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":2,"total":53418655744,"available":53415092224,"internally_reserved":0,"allocated":3563520,"data_stored":3563520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":3,"total":53418655744,"available":53414436864,"internally_reserved":0,"allocated":4218880,"data_stored":4218880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:32:12.468 DEBUG:teuthology.orchestra.run.trial012:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-26T03:32:12.543 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:32:12.538+0000 7f6debe1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:32:12.546 INFO:teuthology.orchestra.run.trial012.stderr:2026-02-26T03:32:12.544+0000 7f6debe1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:32:12.672 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-26T03:32:12.672 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-26T03:32:12.672 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-26T03:32:12.673 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:32:12.737 INFO:tasks.ceph.osd.0:Stopped 2026-02-26T03:32:12.737 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-26T03:32:12.737 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:32:12.807 INFO:tasks.ceph.osd.1:Stopped 2026-02-26T03:32:12.808 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-26T03:32:12.808 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:32:12.878 INFO:tasks.ceph.osd.2:Stopped 2026-02-26T03:32:12.879 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-26T03:32:12.879 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:32:12.944 INFO:tasks.ceph.osd.3:Stopped 2026-02-26T03:32:12.945 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-26T03:32:12.945 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-26T03:32:12.945 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:32:12.978 INFO:tasks.ceph.mgr.x:Stopped 2026-02-26T03:32:12.978 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-26T03:32:12.979 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-26T03:32:12.979 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:32:12.989 INFO:tasks.ceph.mon.a:Stopped 2026-02-26T03:32:12.990 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-26T03:32:12.990 DEBUG:teuthology.orchestra.run.trial012:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v 'but it is still running' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSDMAP_FLAGS\)' | grep -E -v '\(PG_' | grep -E -v '\(OSD_' | grep -E -v '\(OBJECT_' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2026-02-26T03:32:13.057 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial012.front.sepia.ceph.com 2026-02-26T03:32:13.057 DEBUG:teuthology.orchestra.run.trial012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-26T03:32:13.125 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial012.front.sepia.ceph.com 2026-02-26T03:32:13.125 DEBUG:teuthology.orchestra.run.trial012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-26T03:32:13.193 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial068.front.sepia.ceph.com 2026-02-26T03:32:13.193 DEBUG:teuthology.orchestra.run.trial068:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-26T03:32:13.234 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial068.front.sepia.ceph.com 2026-02-26T03:32:13.234 DEBUG:teuthology.orchestra.run.trial068:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-26T03:32:13.300 INFO:tasks.ceph:Archiving mon data... 2026-02-26T03:32:13.301 DEBUG:teuthology.misc:Transferring archived files from trial012:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71166/data/mon.a.tgz 2026-02-26T03:32:13.302 DEBUG:teuthology.orchestra.run.trial012:> mktemp 2026-02-26T03:32:13.318 INFO:teuthology.orchestra.run.trial012.stdout:/tmp/tmp.zoJnjSXz3E 2026-02-26T03:32:13.319 DEBUG:teuthology.orchestra.run.trial012:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.zoJnjSXz3E 2026-02-26T03:32:13.433 DEBUG:teuthology.orchestra.run.trial012:> sudo chmod 0666 /tmp/tmp.zoJnjSXz3E 2026-02-26T03:32:13.515 DEBUG:teuthology.orchestra.remote:trial012:/tmp/tmp.zoJnjSXz3E is 403KB 2026-02-26T03:32:13.610 DEBUG:teuthology.orchestra.run.trial012:> rm -fr /tmp/tmp.zoJnjSXz3E 2026-02-26T03:32:13.627 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-26T03:32:13.627 DEBUG:teuthology.orchestra.run.trial012:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-26T03:32:13.669 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-26T03:32:13.747 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-26T03:32:13.765 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-26T03:32:13.765 INFO:tasks.ceph:Archiving crash dumps... 2026-02-26T03:32:13.768 DEBUG:teuthology.misc:Transferring archived files from trial012:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71166/remote/trial012/crash 2026-02-26T03:32:13.769 DEBUG:teuthology.orchestra.run.trial012:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-26T03:32:13.811 DEBUG:teuthology.misc:Transferring archived files from trial068:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71166/remote/trial068/crash 2026-02-26T03:32:13.812 DEBUG:teuthology.orchestra.run.trial068:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-26T03:32:13.838 INFO:tasks.ceph:Compressing logs... 2026-02-26T03:32:13.838 DEBUG:teuthology.orchestra.run.trial012:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-26T03:32:13.856 DEBUG:teuthology.orchestra.run.trial068:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-26T03:32:13.894 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-26T03:32:13.894 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41176.log 2026-02-26T03:32:13.894 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41568.log 2026-02-26T03:32:13.894 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41176.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-26T03:32:13.894 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41176.log.gz 2026-02-26T03:32:13.894 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41568.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41568.log.gz 2026-02-26T03:32:13.895 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39029.log 2026-02-26T03:32:13.895 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38441.log 2026-02-26T03:32:13.895 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.39029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38911.log 2026-02-26T03:32:13.895 INFO:teuthology.orchestra.run.trial012.stderr: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.39029.log.gz 2026-02-26T03:32:13.895 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41274.log 2026-02-26T03:32:13.895 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38441.log.gz 2026-02-26T03:32:13.895 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37913.log 2026-02-26T03:32:13.895 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38911.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38911.log.gz 2026-02-26T03:32:13.895 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41274.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.36770.log 2026-02-26T03:32:13.895 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41274.log.gz 2026-02-26T03:32:13.895 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.37913.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37913.log.gz 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39426.log 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph.tmp-client.admin.36770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38588.log 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr: 66.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.36770.log.gz 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.39426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38368.log 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.39426.log.gz 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38718.log 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.38588.log.gz 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39078.log 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.38368.log.gz 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41716.log 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38718.log.gz 2026-02-26T03:32:13.896 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.39078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41078.log 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39078.log.gz 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41127.log 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41716.log.gz 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41763.log 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41078.log.gz 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41127.log: gzip -5 --verbose 49.6% -- -- replaced with /var/log/ceph/ceph-client.admin.41127.log.gz /var/log/ceph/ceph.audit.log 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr: 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41763.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41763.log.gz 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41421.log 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38168.log 2026-02-26T03:32:13.897 INFO:teuthology.orchestra.run.trial012.stderr: 88.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38271.log 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41421.log.gz 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38490.log 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38168.log.gz 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39401.log 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38271.log.gz 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38194.log 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38490.log.gz 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.39401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41372.log 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39401.log.gz 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38716.log 2026-02-26T03:32:13.898 INFO:teuthology.orchestra.run.trial012.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.38194.log.gz 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38715.log 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41372.log.gz 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38716.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38716.log.gz 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38717.log 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41519.log 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38715.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38715.log.gz 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38717.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38717.log.gz 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41470.log 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41617.log 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41519.log.gz 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38913.log 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41470.log.gz 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41617.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41617.log.gz 2026-02-26T03:32:13.899 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38539.log 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37960.log 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38913.log.gz 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38539.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.38539.log.gz 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.37960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41666.log 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37960.log.gz 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41225.log 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38912.log 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41666.log.gz 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41225.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41225.log.gz 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39127.log 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38341.log 2026-02-26T03:32:13.900 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38912.log.gz 2026-02-26T03:32:13.901 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.39127.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39127.log.gz 2026-02-26T03:32:13.901 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37827.log 2026-02-26T03:32:13.901 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41323.log 2026-02-26T03:32:13.901 INFO:teuthology.orchestra.run.trial012.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38341.log.gz 2026-02-26T03:32:13.901 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.37827.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37827.log.gz 2026-02-26T03:32:13.901 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38914.log 2026-02-26T03:32:13.901 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.41323.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41323.log.gz 2026-02-26T03:32:13.901 INFO:teuthology.orchestra.run.trial012.stderr:/var/log/ceph/ceph-client.admin.38914.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38914.log.gz 2026-02-26T03:32:13.906 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-26T03:32:13.908 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36924.log 2026-02-26T03:32:13.908 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-26T03:32:13.909 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.36924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36828.log 2026-02-26T03:32:13.909 INFO:teuthology.orchestra.run.trial068.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.36924.log.gz 2026-02-26T03:32:13.909 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36875.log 2026-02-26T03:32:13.909 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.36828.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-26T03:32:13.909 INFO:teuthology.orchestra.run.trial068.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36828.log.gz 2026-02-26T03:32:13.909 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.36875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36980.log 2026-02-26T03:32:13.909 INFO:teuthology.orchestra.run.trial068.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36875.log.gz 2026-02-26T03:32:13.910 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.36980.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36980.log.gz 2026-02-26T03:32:13.933 INFO:teuthology.orchestra.run.trial068.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-26T03:32:14.055 INFO:teuthology.orchestra.run.trial012.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-26T03:32:16.606 INFO:teuthology.orchestra.run.trial012.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-26T03:32:16.680 INFO:teuthology.orchestra.run.trial068.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-26T03:32:16.692 INFO:teuthology.orchestra.run.trial012.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-26T03:32:16.693 INFO:teuthology.orchestra.run.trial012.stderr: 2026-02-26T03:32:16.693 INFO:teuthology.orchestra.run.trial012.stderr:real 0m2.824s 2026-02-26T03:32:16.693 INFO:teuthology.orchestra.run.trial012.stderr:user 0m5.422s 2026-02-26T03:32:16.693 INFO:teuthology.orchestra.run.trial012.stderr:sys 0m0.327s 2026-02-26T03:32:16.708 INFO:teuthology.orchestra.run.trial068.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-26T03:32:16.710 INFO:teuthology.orchestra.run.trial068.stderr: 2026-02-26T03:32:16.710 INFO:teuthology.orchestra.run.trial068.stderr:real 0m2.811s 2026-02-26T03:32:16.710 INFO:teuthology.orchestra.run.trial068.stderr:user 0m5.251s 2026-02-26T03:32:16.710 INFO:teuthology.orchestra.run.trial068.stderr:sys 0m0.357s 2026-02-26T03:32:16.711 INFO:tasks.ceph:Archiving logs... 2026-02-26T03:32:16.711 DEBUG:teuthology.misc:Transferring archived files from trial012:/var/log/ceph to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71166/remote/trial012/log 2026-02-26T03:32:16.713 DEBUG:teuthology.orchestra.run.trial012:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-26T03:32:17.311 DEBUG:teuthology.misc:Transferring archived files from trial068:/var/log/ceph to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71166/remote/trial068/log 2026-02-26T03:32:17.311 DEBUG:teuthology.orchestra.run.trial068:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-26T03:32:17.834 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-26T03:32:17.958 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-26T03:32:17.970 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-26T03:32:17.970 DEBUG:teuthology.orchestra.run.trial012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-26T03:32:17.972 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-26T03:32:17.998 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-26T03:32:18.009 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-26T03:32:18.010 DEBUG:teuthology.orchestra.run.trial012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-26T03:32:18.039 DEBUG:teuthology.orchestra.run.trial068:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-26T03:32:18.051 INFO:teuthology.orchestra.run.trial012.stderr:bash: line 1: ntpq: command not found 2026-02-26T03:32:18.054 INFO:teuthology.orchestra.run.trial068.stderr:bash: line 1: ntpq: command not found 2026-02-26T03:32:18.086 INFO:teuthology.orchestra.run.trial012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-26T03:32:18.086 INFO:teuthology.orchestra.run.trial012.stdout:=============================================================================== 2026-02-26T03:32:18.086 INFO:teuthology.orchestra.run.trial012.stdout:^+ 23-95-35-34-host.colocro> 2 7 377 59 -679us[ -679us] +/- 36ms 2026-02-26T03:32:18.086 INFO:teuthology.orchestra.run.trial012.stdout:^+ 23.186.168.131 2 7 137 50 -34us[ -34us] +/- 56ms 2026-02-26T03:32:18.087 INFO:teuthology.orchestra.run.trial012.stdout:^* B1-66ER.matrix.gs 2 7 377 121 -3428ns[ -27us] +/- 47ms 2026-02-26T03:32:18.087 INFO:teuthology.orchestra.run.trial012.stdout:^+ 66.42.71.197.vultruserco> 2 6 377 58 +204us[ +204us] +/- 47ms 2026-02-26T03:32:18.088 INFO:teuthology.orchestra.run.trial068.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-26T03:32:18.088 INFO:teuthology.orchestra.run.trial068.stdout:=============================================================================== 2026-02-26T03:32:18.088 INFO:teuthology.orchestra.run.trial068.stdout:^+ 172-234-37-140.ip.linode> 2 6 377 60 +17ms[ +17ms] +/- 95ms 2026-02-26T03:32:18.088 INFO:teuthology.orchestra.run.trial068.stdout:^* time.trtnw.net 2 6 377 59 -3178us[-3045us] +/- 36ms 2026-02-26T03:32:18.088 INFO:teuthology.orchestra.run.trial068.stdout:^- ip216-250-115-174.pbiaas> 2 7 377 124 -9771us[-9601us] +/- 106ms 2026-02-26T03:32:18.088 INFO:teuthology.orchestra.run.trial068.stdout:^+ y.ns.gin.ntt.net 2 7 377 122 -3042us[-2900us] +/- 47ms 2026-02-26T03:32:18.088 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-26T03:32:18.101 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-26T03:32:18.101 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-26T03:32:18.115 DEBUG:teuthology.orchestra.run.trial012:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-26T03:32:18.181 DEBUG:teuthology.orchestra.run.trial068:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-26T03:32:18.235 DEBUG:teuthology.orchestra.run.trial012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-26T03:32:18.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:32:18.265 DEBUG:teuthology.orchestra.run.trial068:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-26T03:32:18.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:32:18.291 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-26T03:32:18.306 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-26T03:32:18.324 INFO:teuthology.task.internal:Duration was 1396.701740 seconds 2026-02-26T03:32:18.325 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-26T03:32:18.341 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-26T03:32:18.341 DEBUG:teuthology.orchestra.run.trial012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-26T03:32:18.344 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-26T03:32:18.379 INFO:teuthology.orchestra.run.trial012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-26T03:32:18.381 INFO:teuthology.orchestra.run.trial068.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-26T03:32:18.547 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-26T03:32:18.547 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial012.front.sepia.ceph.com 2026-02-26T03:32:18.547 DEBUG:teuthology.orchestra.run.trial012:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-26T03:32:18.605 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial068.front.sepia.ceph.com 2026-02-26T03:32:18.605 DEBUG:teuthology.orchestra.run.trial068:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-26T03:32:18.627 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-26T03:32:18.627 DEBUG:teuthology.orchestra.run.trial012:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-26T03:32:18.648 DEBUG:teuthology.orchestra.run.trial068:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-26T03:32:18.726 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-26T03:32:18.727 DEBUG:teuthology.orchestra.run.trial012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-26T03:32:18.729 DEBUG:teuthology.orchestra.run.trial068:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-26T03:32:18.750 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:32:18.751 INFO:teuthology.orchestra.run.trial012.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:32:18.751 INFO:teuthology.orchestra.run.trial012.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-26T03:32:18.751 INFO:teuthology.orchestra.run.trial012.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-26T03:32:18.751 INFO:teuthology.orchestra.run.trial012.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-26T03:32:18.752 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:32:18.753 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:32:18.754 INFO:teuthology.orchestra.run.trial068.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-26T03:32:18.754 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-26T03:32:18.754 INFO:teuthology.orchestra.run.trial068.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-26T03:32:18.760 INFO:teuthology.orchestra.run.trial012.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-26T03:32:18.764 INFO:teuthology.orchestra.run.trial068.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.6% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-26T03:32:18.766 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-26T03:32:18.782 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-26T03:32:18.783 DEBUG:teuthology.orchestra.run.trial012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-26T03:32:18.827 DEBUG:teuthology.orchestra.run.trial068:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-26T03:32:18.852 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-26T03:32:18.869 DEBUG:teuthology.orchestra.run.trial012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:32:18.872 DEBUG:teuthology.orchestra.run.trial068:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:32:18.892 INFO:teuthology.orchestra.run.trial012.stdout:kernel.core_pattern = core 2026-02-26T03:32:18.915 INFO:teuthology.orchestra.run.trial068.stdout:kernel.core_pattern = core 2026-02-26T03:32:18.927 DEBUG:teuthology.orchestra.run.trial012:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:32:18.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:32:18.962 DEBUG:teuthology.orchestra.run.trial068:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:32:18.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:32:18.982 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-26T03:32:19.000 INFO:teuthology.task.internal:Transferring archived files... 2026-02-26T03:32:19.001 DEBUG:teuthology.misc:Transferring archived files from trial012:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71166/remote/trial012 2026-02-26T03:32:19.001 DEBUG:teuthology.orchestra.run.trial012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-26T03:32:19.037 DEBUG:teuthology.misc:Transferring archived files from trial068:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71166/remote/trial068 2026-02-26T03:32:19.038 DEBUG:teuthology.orchestra.run.trial068:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-26T03:32:19.074 INFO:teuthology.task.internal:Removing archive directory... 2026-02-26T03:32:19.074 DEBUG:teuthology.orchestra.run.trial012:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-26T03:32:19.077 DEBUG:teuthology.orchestra.run.trial068:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-26T03:32:19.121 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-26T03:32:19.140 INFO:teuthology.task.internal:Not uploading archives. 2026-02-26T03:32:19.140 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-26T03:32:19.157 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-26T03:32:19.157 DEBUG:teuthology.orchestra.run.trial012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-26T03:32:19.160 DEBUG:teuthology.orchestra.run.trial068:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-26T03:32:19.175 INFO:teuthology.orchestra.run.trial012.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 03:32 /home/ubuntu/cephtest 2026-02-26T03:32:19.177 INFO:teuthology.orchestra.run.trial068.stdout: 917521 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 03:32 /home/ubuntu/cephtest 2026-02-26T03:32:19.179 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-26T03:32:19.197 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-26T03:32:19.237 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rados_python} duration: 1396.7017402648926 flavor: debug owner: scheduled_shraddhaag@soko04.front.sepia.ceph.com success: true 2026-02-26T03:32:19.238 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-26T03:32:19.355 INFO:teuthology.run:pass