2026-02-26T03:42:02.282 INFO:root:teuthology version: 1.2.4.dev11+g6e3d0419c 2026-02-26T03:42:02.301 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-26T03:42:02.379 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71262 branch: main description: crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/rados_python} email: null first_in_suite: false flavor: debug job_id: '71262' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(PG_ - \(OSD_ - \(OBJECT_ - \(POOL_APP_NOT_ENABLED\) sha1: abffb617683951511fccf3c3cab6e219f2ec83a8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_system_packages: - python3-pytest flavor: debug sha1: abffb617683951511fccf3c3cab6e219f2ec83a8 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996 owner: scheduled_shraddhaag@soko04.front.sepia.ceph.com priority: 101 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 5117 sha1: abffb617683951511fccf3c3cab6e219f2ec83a8 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996 targets: trial160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAmKoMZxpPU5T/Bxq+g9wCBUqaaDDrIIcYb15cINtnZ1/IWb/NZ2PmjEABs3NWcoKUjOhdh2PYGMEcMdXyBJrLU= trial195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC9C7nCXyYm8D8zvsOtcO6RiBr8aoxp5NEqzbJfdx5dtzsZdhTqOM5G0vbA7nwTDtdiSgpuy/b6p7GvF1bjCxDU= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 - workunit: clients: client.0: - rados/test_python.sh -m 'not (wait or tier or ec)' timeout: 1h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6e3d0419ccca0e75a03e08feb708946e198236f4 timestamp: 2026-02-26_02:22:56 tube: trial user: shraddhaag verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-26T03:42:02.380 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996/qa; will attempt to use it 2026-02-26T03:42:02.380 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996/qa/tasks 2026-02-26T03:42:02.380 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-26T03:42:02.382 INFO:teuthology.task.internal:Checking packages... 2026-02-26T03:42:02.382 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash 'abffb617683951511fccf3c3cab6e219f2ec83a8' 2026-02-26T03:42:02.382 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-26T03:42:02.382 INFO:teuthology.packaging:ref: None 2026-02-26T03:42:02.382 INFO:teuthology.packaging:tag: None 2026-02-26T03:42:02.382 INFO:teuthology.packaging:branch: main 2026-02-26T03:42:02.382 INFO:teuthology.packaging:sha1: abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:42:02.382 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=main 2026-02-26T03:42:02.507 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5603.g237566a0 2026-02-26T03:42:02.509 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-26T03:42:02.520 INFO:teuthology.task.internal:no buildpackages task found 2026-02-26T03:42:02.520 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-26T03:42:02.533 INFO:teuthology.task.internal:Saving configuration 2026-02-26T03:42:02.545 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-26T03:42:02.557 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-26T03:42:02.637 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial160.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71262', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-26 03:38:36.270270', 'locked_by': 'scheduled_shraddhaag@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAmKoMZxpPU5T/Bxq+g9wCBUqaaDDrIIcYb15cINtnZ1/IWb/NZ2PmjEABs3NWcoKUjOhdh2PYGMEcMdXyBJrLU='} 2026-02-26T03:42:02.693 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial195.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71262', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-26 03:38:36.270915', 'locked_by': 'scheduled_shraddhaag@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC9C7nCXyYm8D8zvsOtcO6RiBr8aoxp5NEqzbJfdx5dtzsZdhTqOM5G0vbA7nwTDtdiSgpuy/b6p7GvF1bjCxDU='} 2026-02-26T03:42:02.693 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-26T03:42:02.704 INFO:teuthology.task.internal:roles: ubuntu@trial160.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2026-02-26T03:42:02.704 INFO:teuthology.task.internal:roles: ubuntu@trial195.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2026-02-26T03:42:02.704 INFO:teuthology.run_tasks:Running task console_log... 2026-02-26T03:42:02.838 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x744f02a1ed40>, signals=[15]) 2026-02-26T03:42:02.838 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-26T03:42:02.850 INFO:teuthology.task.internal:Opening connections... 2026-02-26T03:42:02.850 DEBUG:teuthology.task.internal:connecting to ubuntu@trial160.front.sepia.ceph.com 2026-02-26T03:42:02.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:42:02.911 DEBUG:teuthology.task.internal:connecting to ubuntu@trial195.front.sepia.ceph.com 2026-02-26T03:42:02.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:42:02.973 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-26T03:42:02.985 DEBUG:teuthology.orchestra.run.trial160:> uname -m 2026-02-26T03:42:03.001 INFO:teuthology.orchestra.run.trial160.stdout:x86_64 2026-02-26T03:42:03.001 DEBUG:teuthology.orchestra.run.trial160:> cat /etc/os-release 2026-02-26T03:42:03.056 INFO:teuthology.orchestra.run.trial160.stdout:NAME="CentOS Stream" 2026-02-26T03:42:03.056 INFO:teuthology.orchestra.run.trial160.stdout:VERSION="9" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:ID="centos" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:ID_LIKE="rhel fedora" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:VERSION_ID="9" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:PLATFORM_ID="platform:el9" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:ANSI_COLOR="0;31" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:LOGO="fedora-logo-icon" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:HOME_URL="https://centos.org/" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-26T03:42:03.057 INFO:teuthology.orchestra.run.trial160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-26T03:42:03.057 INFO:teuthology.lock.ops:Updating trial160.front.sepia.ceph.com on lock server 2026-02-26T03:42:03.131 DEBUG:teuthology.orchestra.run.trial195:> uname -m 2026-02-26T03:42:03.146 INFO:teuthology.orchestra.run.trial195.stdout:x86_64 2026-02-26T03:42:03.146 DEBUG:teuthology.orchestra.run.trial195:> cat /etc/os-release 2026-02-26T03:42:03.201 INFO:teuthology.orchestra.run.trial195.stdout:NAME="CentOS Stream" 2026-02-26T03:42:03.201 INFO:teuthology.orchestra.run.trial195.stdout:VERSION="9" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:ID="centos" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:ID_LIKE="rhel fedora" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:VERSION_ID="9" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:PLATFORM_ID="platform:el9" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:ANSI_COLOR="0;31" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:LOGO="fedora-logo-icon" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:HOME_URL="https://centos.org/" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-26T03:42:03.202 INFO:teuthology.orchestra.run.trial195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-26T03:42:03.202 INFO:teuthology.lock.ops:Updating trial195.front.sepia.ceph.com on lock server 2026-02-26T03:42:03.279 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-26T03:42:03.293 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-26T03:42:03.303 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-26T03:42:03.304 DEBUG:teuthology.orchestra.run.trial160:> test '!' -e /home/ubuntu/cephtest 2026-02-26T03:42:03.305 DEBUG:teuthology.orchestra.run.trial195:> test '!' -e /home/ubuntu/cephtest 2026-02-26T03:42:03.320 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-26T03:42:03.330 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-26T03:42:03.330 DEBUG:teuthology.orchestra.run.trial160:> test -z $(ls -A /var/lib/ceph) 2026-02-26T03:42:03.361 DEBUG:teuthology.orchestra.run.trial195:> test -z $(ls -A /var/lib/ceph) 2026-02-26T03:42:03.378 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-26T03:42:03.534 INFO:teuthology.run_tasks:Running task kernel... 2026-02-26T03:42:03.554 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-26T03:42:03.555 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-26T03:42:03.555 DEBUG:teuthology.orchestra.run.trial160:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-26T03:42:03.555 DEBUG:teuthology.orchestra.run.trial195:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-26T03:42:03.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:42:03.570 DEBUG:teuthology.orchestra.run.trial195:> uname -r 2026-02-26T03:42:03.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:42:03.570 DEBUG:teuthology.orchestra.run.trial160:> uname -r 2026-02-26T03:42:03.626 INFO:teuthology.orchestra.run.trial195.stdout:5.14.0-681.el9.x86_64 2026-02-26T03:42:03.626 INFO:teuthology.task.kernel:Running kernel on trial195: 5.14.0-681.el9.x86_64 2026-02-26T03:42:03.626 DEBUG:teuthology.orchestra.run.trial195:> sudo yum install -y kernel 2026-02-26T03:42:03.626 INFO:teuthology.orchestra.run.trial160.stdout:5.14.0-681.el9.x86_64 2026-02-26T03:42:03.626 INFO:teuthology.task.kernel:Running kernel on trial160: 5.14.0-681.el9.x86_64 2026-02-26T03:42:03.627 DEBUG:teuthology.orchestra.run.trial160:> sudo yum install -y kernel 2026-02-26T03:42:04.263 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - BaseOS 35 MB/s | 8.9 MB 00:00 2026-02-26T03:42:04.530 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.9 MB 00:00 2026-02-26T03:42:07.536 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 27 MB 00:02 2026-02-26T03:42:08.460 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 27 MB 00:03 2026-02-26T03:42:10.975 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 8.0 MB 00:01 2026-02-26T03:42:11.943 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 8.0 MB 00:01 2026-02-26T03:42:12.161 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 20 kB 00:00 2026-02-26T03:42:13.371 INFO:teuthology.orchestra.run.trial160.stdout:Extra Packages for Enterprise Linux 18 MB/s | 20 MB 00:01 2026-02-26T03:42:16.547 INFO:teuthology.orchestra.run.trial160.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-26T03:42:17.568 INFO:teuthology.orchestra.run.trial160.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-26T03:42:17.568 INFO:teuthology.orchestra.run.trial160.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-26T03:42:17.587 INFO:teuthology.orchestra.run.trial160.stdout:Dependencies resolved. 2026-02-26T03:42:17.589 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================ 2026-02-26T03:42:17.589 INFO:teuthology.orchestra.run.trial160.stdout: Package Architecture Version Repository Size 2026-02-26T03:42:17.589 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================ 2026-02-26T03:42:17.589 INFO:teuthology.orchestra.run.trial160.stdout:Installing: 2026-02-26T03:42:17.589 INFO:teuthology.orchestra.run.trial160.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-26T03:42:17.589 INFO:teuthology.orchestra.run.trial160.stdout:Installing dependencies: 2026-02-26T03:42:17.589 INFO:teuthology.orchestra.run.trial160.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-26T03:42:17.589 INFO:teuthology.orchestra.run.trial160.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-26T03:42:17.589 INFO:teuthology.orchestra.run.trial160.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-26T03:42:17.589 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:42:17.590 INFO:teuthology.orchestra.run.trial160.stdout:Transaction Summary 2026-02-26T03:42:17.590 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================ 2026-02-26T03:42:17.590 INFO:teuthology.orchestra.run.trial160.stdout:Install 4 Packages 2026-02-26T03:42:17.590 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:42:17.590 INFO:teuthology.orchestra.run.trial160.stdout:Total download size: 89 M 2026-02-26T03:42:17.590 INFO:teuthology.orchestra.run.trial160.stdout:Installed size: 135 M 2026-02-26T03:42:17.590 INFO:teuthology.orchestra.run.trial160.stdout:Downloading Packages: 2026-02-26T03:42:17.776 INFO:teuthology.orchestra.run.trial160.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 6.9 MB/s | 901 kB 00:00 2026-02-26T03:42:17.999 INFO:teuthology.orchestra.run.trial160.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 49 MB/s | 17 MB 00:00 2026-02-26T03:42:18.186 INFO:teuthology.orchestra.run.trial160.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 76 MB/s | 31 MB 00:00 2026-02-26T03:42:18.300 INFO:teuthology.orchestra.run.trial160.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 61 MB/s | 40 MB 00:00 2026-02-26T03:42:18.301 INFO:teuthology.orchestra.run.trial160.stdout:-------------------------------------------------------------------------------- 2026-02-26T03:42:18.301 INFO:teuthology.orchestra.run.trial160.stdout:Total 125 MB/s | 89 MB 00:00 2026-02-26T03:42:18.492 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction check 2026-02-26T03:42:18.702 INFO:teuthology.orchestra.run.trial160.stdout:Transaction check succeeded. 2026-02-26T03:42:18.702 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction test 2026-02-26T03:42:19.084 INFO:teuthology.orchestra.run.trial160.stdout:Transaction test succeeded. 2026-02-26T03:42:19.085 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction 2026-02-26T03:42:20.199 INFO:teuthology.orchestra.run.trial160.stdout: Preparing : 1/1 2026-02-26T03:42:20.491 INFO:teuthology.orchestra.run.trial160.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-26T03:42:20.513 INFO:teuthology.orchestra.run.trial160.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:42:23.092 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:42:23.125 INFO:teuthology.orchestra.run.trial160.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:42:26.380 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:42:26.385 INFO:teuthology.orchestra.run.trial160.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:42:29.665 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:42:47.247 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:42:47.271 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:42:47.459 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:42:47.459 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-26T03:42:47.459 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:42:47.459 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:42:47.516 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:42:47.516 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:42:47.516 INFO:teuthology.orchestra.run.trial160.stdout:Installed: 2026-02-26T03:42:47.516 INFO:teuthology.orchestra.run.trial160.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-26T03:42:47.517 INFO:teuthology.orchestra.run.trial160.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-26T03:42:47.517 INFO:teuthology.orchestra.run.trial160.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-26T03:42:47.517 INFO:teuthology.orchestra.run.trial160.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-26T03:42:47.517 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:42:47.517 INFO:teuthology.orchestra.run.trial160.stdout:Complete! 2026-02-26T03:42:47.580 DEBUG:teuthology.orchestra.run.trial160:> rpm -q kernel | sort -rV | head -n 1 2026-02-26T03:42:47.609 INFO:teuthology.orchestra.run.trial160.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-26T03:42:47.610 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-26T03:42:47.610 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-26T03:42:47.610 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-26T03:42:47.610 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-26T03:42:47.611 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-26T03:42:47.611 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial160.front.sepia.ceph.com, path=None, version=distro) 2026-02-26T03:42:47.611 DEBUG:teuthology.orchestra.run.trial160:> rpm -q kernel | sort -rV | head -n 1 2026-02-26T03:42:47.676 INFO:teuthology.orchestra.run.trial160.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-26T03:42:47.678 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-26T03:42:47.678 DEBUG:teuthology.orchestra.run.trial160:> sudo rpm -qi grub2-tools 2026-02-26T03:42:47.747 INFO:teuthology.orchestra.run.trial160.stdout:Name : grub2-tools 2026-02-26T03:42:47.747 INFO:teuthology.orchestra.run.trial160.stdout:Epoch : 1 2026-02-26T03:42:47.747 INFO:teuthology.orchestra.run.trial160.stdout:Version : 2.06 2026-02-26T03:42:47.747 INFO:teuthology.orchestra.run.trial160.stdout:Release : 120.el9 2026-02-26T03:42:47.747 INFO:teuthology.orchestra.run.trial160.stdout:Architecture: x86_64 2026-02-26T03:42:47.747 INFO:teuthology.orchestra.run.trial160.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:Group : Unspecified 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:Size : 9108611 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:License : GPLv3+ 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:Packager : builder@centos.org 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:Vendor : CentOS 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:Summary : Support tools for GRUB. 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout:Description : 2026-02-26T03:42:47.748 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:42:47.749 INFO:teuthology.orchestra.run.trial160.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-26T03:42:47.749 INFO:teuthology.orchestra.run.trial160.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-26T03:42:47.749 INFO:teuthology.orchestra.run.trial160.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-26T03:42:47.749 INFO:teuthology.orchestra.run.trial160.stdout:hardware devices. 2026-02-26T03:42:47.749 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:42:47.749 INFO:teuthology.orchestra.run.trial160.stdout:This subpackage provides tools for support of all platforms. 2026-02-26T03:42:47.750 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-26T03:42:47.750 INFO:teuthology.task.kernel:Updating grub on trial160 to boot 5.14.0-686.el9.x86_64 2026-02-26T03:42:47.750 DEBUG:teuthology.orchestra.run.trial160:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-26T03:42:47.815 INFO:teuthology.orchestra.run.trial160.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-26T03:42:47.817 DEBUG:teuthology.orchestra.run.trial160:> sudo command -v grubby && echo yes || echo no 2026-02-26T03:42:47.881 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/grubby 2026-02-26T03:42:47.883 INFO:teuthology.orchestra.run.trial160.stdout:yes 2026-02-26T03:42:47.883 DEBUG:teuthology.orchestra.run.trial160:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-26T03:42:47.946 INFO:teuthology.orchestra.run.trial160.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-26T03:42:47.947 DEBUG:teuthology.orchestra.run.trial160:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-26T03:42:48.020 DEBUG:teuthology.orchestra.run.trial160:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-26T03:42:48.223 INFO:teuthology.orchestra.run.trial160.stderr:Generating grub configuration file ... 2026-02-26T03:42:48.290 INFO:teuthology.orchestra.run.trial160.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-26T03:42:48.297 INFO:teuthology.orchestra.run.trial160.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-26T03:42:48.545 INFO:teuthology.orchestra.run.trial160.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-26T03:42:48.552 INFO:teuthology.orchestra.run.trial160.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-26T03:42:48.587 INFO:teuthology.orchestra.run.trial160.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-26T03:42:48.594 INFO:teuthology.orchestra.run.trial160.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-26T03:42:48.626 INFO:teuthology.orchestra.run.trial160.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-26T03:42:48.632 INFO:teuthology.orchestra.run.trial160.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-26T03:42:48.656 INFO:teuthology.orchestra.run.trial160.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-26T03:42:48.663 INFO:teuthology.orchestra.run.trial160.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-26T03:42:48.748 INFO:teuthology.orchestra.run.trial160.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-26T03:42:48.757 INFO:teuthology.orchestra.run.trial160.stderr:done 2026-02-26T03:42:48.759 DEBUG:teuthology.orchestra.run.trial160:> test -d /sys/firmware/efi 2026-02-26T03:42:48.775 DEBUG:teuthology.orchestra.run.trial160:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-26T03:42:48.841 INFO:teuthology.orchestra.run.trial160.stdout:/boot/efi/EFI/BOOT 2026-02-26T03:42:48.841 INFO:teuthology.orchestra.run.trial160.stdout:/boot/efi/EFI/centos 2026-02-26T03:42:48.843 DEBUG:teuthology.orchestra.run.trial160:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-26T03:42:48.910 INFO:teuthology.orchestra.run.trial160.stdout:no 2026-02-26T03:42:48.910 DEBUG:teuthology.orchestra.run.trial160:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-26T03:42:48.975 INFO:teuthology.orchestra.run.trial160.stdout:yes 2026-02-26T03:42:48.975 DEBUG:teuthology.orchestra.run.trial160:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-26T03:42:49.039 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial160 2026-02-26T03:42:49.039 DEBUG:teuthology.orchestra.run.trial160:> sudo bash -c ' 2026-02-26T03:42:49.039 DEBUG:teuthology.orchestra.run.trial160:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-26T03:42:49.039 DEBUG:teuthology.orchestra.run.trial160:> ' 2026-02-26T03:43:09.577 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - Extras packages 363 B/s | 20 kB 00:56 2026-02-26T03:43:11.669 INFO:teuthology.orchestra.run.trial195.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:02 2026-02-26T03:43:14.876 INFO:teuthology.orchestra.run.trial195.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-26T03:43:15.895 INFO:teuthology.orchestra.run.trial195.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-26T03:43:15.895 INFO:teuthology.orchestra.run.trial195.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-26T03:43:15.915 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout: Package Architecture Version Repository Size 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout:Install 4 Packages 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 89 M 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 135 M 2026-02-26T03:43:15.916 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-26T03:43:17.427 INFO:teuthology.orchestra.run.trial195.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 621 kB/s | 901 kB 00:01 2026-02-26T03:43:18.442 INFO:teuthology.orchestra.run.trial195.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 7.0 MB/s | 17 MB 00:02 2026-02-26T03:43:19.082 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-26T03:43:19.082 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial160.front.sepia.ceph.com' 2026-02-26T03:43:19.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:43:19.371 INFO:teuthology.orchestra.run.trial195.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 16 MB/s | 31 MB 00:01 2026-02-26T03:43:24.561 INFO:teuthology.orchestra.run.trial195.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 4.6 MB/s | 40 MB 00:08 2026-02-26T03:43:24.563 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-26T03:43:24.563 INFO:teuthology.orchestra.run.trial195.stdout:Total 10 MB/s | 89 MB 00:08 2026-02-26T03:43:24.754 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-26T03:43:24.958 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-26T03:43:24.958 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-26T03:43:25.343 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-26T03:43:25.343 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-26T03:43:26.459 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-26T03:43:26.764 INFO:teuthology.orchestra.run.trial195.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-26T03:43:26.786 INFO:teuthology.orchestra.run.trial195.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:43:29.371 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:43:29.404 INFO:teuthology.orchestra.run.trial195.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:43:32.439 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:43:32.445 INFO:teuthology.orchestra.run.trial195.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:43:35.489 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:43:52.935 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:43:52.957 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:43:53.142 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:43:53.143 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-26T03:43:53.143 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-26T03:43:53.143 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-26T03:43:53.200 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-26T03:43:53.200 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:43:53.200 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-26T03:43:53.201 INFO:teuthology.orchestra.run.trial195.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-26T03:43:53.201 INFO:teuthology.orchestra.run.trial195.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-26T03:43:53.201 INFO:teuthology.orchestra.run.trial195.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-26T03:43:53.201 INFO:teuthology.orchestra.run.trial195.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-26T03:43:53.201 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:43:53.201 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-26T03:43:53.261 DEBUG:teuthology.orchestra.run.trial195:> rpm -q kernel | sort -rV | head -n 1 2026-02-26T03:43:53.290 INFO:teuthology.orchestra.run.trial195.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-26T03:43:53.291 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-26T03:43:53.291 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-26T03:43:53.291 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-26T03:43:53.291 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-26T03:43:53.292 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2026-02-26T03:43:53.292 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial195.front.sepia.ceph.com, path=None, version=distro) 2026-02-26T03:43:53.292 DEBUG:teuthology.orchestra.run.trial195:> rpm -q kernel | sort -rV | head -n 1 2026-02-26T03:43:53.356 INFO:teuthology.orchestra.run.trial195.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-26T03:43:53.357 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-26T03:43:53.357 DEBUG:teuthology.orchestra.run.trial195:> sudo rpm -qi grub2-tools 2026-02-26T03:43:53.427 INFO:teuthology.orchestra.run.trial195.stdout:Name : grub2-tools 2026-02-26T03:43:53.427 INFO:teuthology.orchestra.run.trial195.stdout:Epoch : 1 2026-02-26T03:43:53.427 INFO:teuthology.orchestra.run.trial195.stdout:Version : 2.06 2026-02-26T03:43:53.427 INFO:teuthology.orchestra.run.trial195.stdout:Release : 120.el9 2026-02-26T03:43:53.427 INFO:teuthology.orchestra.run.trial195.stdout:Architecture: x86_64 2026-02-26T03:43:53.427 INFO:teuthology.orchestra.run.trial195.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-26T03:43:53.427 INFO:teuthology.orchestra.run.trial195.stdout:Group : Unspecified 2026-02-26T03:43:53.427 INFO:teuthology.orchestra.run.trial195.stdout:Size : 9108611 2026-02-26T03:43:53.428 INFO:teuthology.orchestra.run.trial195.stdout:License : GPLv3+ 2026-02-26T03:43:53.428 INFO:teuthology.orchestra.run.trial195.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-26T03:43:53.428 INFO:teuthology.orchestra.run.trial195.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-26T03:43:53.428 INFO:teuthology.orchestra.run.trial195.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-26T03:43:53.428 INFO:teuthology.orchestra.run.trial195.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-26T03:43:53.428 INFO:teuthology.orchestra.run.trial195.stdout:Packager : builder@centos.org 2026-02-26T03:43:53.428 INFO:teuthology.orchestra.run.trial195.stdout:Vendor : CentOS 2026-02-26T03:43:53.428 INFO:teuthology.orchestra.run.trial195.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-26T03:43:53.428 INFO:teuthology.orchestra.run.trial195.stdout:Summary : Support tools for GRUB. 2026-02-26T03:43:53.428 INFO:teuthology.orchestra.run.trial195.stdout:Description : 2026-02-26T03:43:53.429 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:43:53.429 INFO:teuthology.orchestra.run.trial195.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-26T03:43:53.429 INFO:teuthology.orchestra.run.trial195.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-26T03:43:53.429 INFO:teuthology.orchestra.run.trial195.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-26T03:43:53.429 INFO:teuthology.orchestra.run.trial195.stdout:hardware devices. 2026-02-26T03:43:53.429 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:43:53.429 INFO:teuthology.orchestra.run.trial195.stdout:This subpackage provides tools for support of all platforms. 2026-02-26T03:43:53.430 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-26T03:43:53.430 INFO:teuthology.task.kernel:Updating grub on trial195 to boot 5.14.0-686.el9.x86_64 2026-02-26T03:43:53.431 DEBUG:teuthology.orchestra.run.trial195:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-26T03:43:53.496 INFO:teuthology.orchestra.run.trial195.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-26T03:43:53.497 DEBUG:teuthology.orchestra.run.trial195:> sudo command -v grubby && echo yes || echo no 2026-02-26T03:43:53.560 INFO:teuthology.orchestra.run.trial195.stdout:/sbin/grubby 2026-02-26T03:43:53.561 INFO:teuthology.orchestra.run.trial195.stdout:yes 2026-02-26T03:43:53.562 DEBUG:teuthology.orchestra.run.trial195:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-26T03:43:53.629 INFO:teuthology.orchestra.run.trial195.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-26T03:43:53.629 DEBUG:teuthology.orchestra.run.trial195:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-26T03:43:53.701 DEBUG:teuthology.orchestra.run.trial195:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-26T03:43:53.909 INFO:teuthology.orchestra.run.trial195.stderr:Generating grub configuration file ... 2026-02-26T03:43:53.978 INFO:teuthology.orchestra.run.trial195.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-26T03:43:53.985 INFO:teuthology.orchestra.run.trial195.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-26T03:43:54.236 INFO:teuthology.orchestra.run.trial195.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-26T03:43:54.243 INFO:teuthology.orchestra.run.trial195.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-26T03:43:54.277 INFO:teuthology.orchestra.run.trial195.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-26T03:43:54.283 INFO:teuthology.orchestra.run.trial195.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-26T03:43:54.315 INFO:teuthology.orchestra.run.trial195.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-26T03:43:54.322 INFO:teuthology.orchestra.run.trial195.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-26T03:43:54.345 INFO:teuthology.orchestra.run.trial195.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-26T03:43:54.351 INFO:teuthology.orchestra.run.trial195.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-26T03:43:54.437 INFO:teuthology.orchestra.run.trial195.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-26T03:43:54.446 INFO:teuthology.orchestra.run.trial195.stderr:done 2026-02-26T03:43:54.448 DEBUG:teuthology.orchestra.run.trial195:> test -d /sys/firmware/efi 2026-02-26T03:43:54.464 DEBUG:teuthology.orchestra.run.trial195:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-26T03:43:54.530 INFO:teuthology.orchestra.run.trial195.stdout:/boot/efi/EFI/BOOT 2026-02-26T03:43:54.530 INFO:teuthology.orchestra.run.trial195.stdout:/boot/efi/EFI/centos 2026-02-26T03:43:54.531 DEBUG:teuthology.orchestra.run.trial195:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-26T03:43:54.597 INFO:teuthology.orchestra.run.trial195.stdout:no 2026-02-26T03:43:54.597 DEBUG:teuthology.orchestra.run.trial195:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-26T03:43:54.660 INFO:teuthology.orchestra.run.trial195.stdout:yes 2026-02-26T03:43:54.661 DEBUG:teuthology.orchestra.run.trial195:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-26T03:43:54.725 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial195 2026-02-26T03:43:54.725 DEBUG:teuthology.orchestra.run.trial195:> sudo bash -c ' 2026-02-26T03:43:54.725 DEBUG:teuthology.orchestra.run.trial195:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-26T03:43:54.726 DEBUG:teuthology.orchestra.run.trial195:> ' 2026-02-26T03:44:19.085 DEBUG:teuthology.orchestra.remote:timed out 2026-02-26T03:44:24.768 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2026-02-26T03:44:24.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial195.front.sepia.ceph.com' 2026-02-26T03:44:24.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:44:28.090 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial160.front.sepia.ceph.com' 2026-02-26T03:44:28.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:44:28.221 DEBUG:teuthology.orchestra.run.trial160:> true 2026-02-26T03:44:28.376 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial160.front.sepia.ceph.com' 2026-02-26T03:44:28.376 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-686.el9.x86_64"... 2026-02-26T03:44:28.377 DEBUG:teuthology.orchestra.run.trial160:> uname -r 2026-02-26T03:44:28.433 INFO:teuthology.orchestra.run.trial160.stdout:5.14.0-686.el9.x86_64 2026-02-26T03:44:28.433 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-26T03:44:28.434 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-26T03:44:28.434 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-26T03:44:29.435 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-26T03:44:29.435 DEBUG:teuthology.orchestra.run.trial160:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-26T03:44:29.465 INFO:teuthology.orchestra.run.trial160.stdout:ttyS1 2026-02-26T03:44:29.482 DEBUG:teuthology.parallel:result is None 2026-02-26T03:45:24.771 DEBUG:teuthology.orchestra.remote:timed out 2026-02-26T03:45:33.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial195.front.sepia.ceph.com' 2026-02-26T03:45:33.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:45:33.905 DEBUG:teuthology.orchestra.run.trial195:> true 2026-02-26T03:45:34.070 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial195.front.sepia.ceph.com' 2026-02-26T03:45:34.070 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-686.el9.x86_64"... 2026-02-26T03:45:34.071 DEBUG:teuthology.orchestra.run.trial195:> uname -r 2026-02-26T03:45:34.127 INFO:teuthology.orchestra.run.trial195.stdout:5.14.0-686.el9.x86_64 2026-02-26T03:45:34.128 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-26T03:45:34.128 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-26T03:45:34.129 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-26T03:45:35.130 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2026-02-26T03:45:35.130 DEBUG:teuthology.orchestra.run.trial195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-26T03:45:35.160 INFO:teuthology.orchestra.run.trial195.stdout:ttyS1 2026-02-26T03:45:35.180 DEBUG:teuthology.parallel:result is None 2026-02-26T03:45:35.180 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-26T03:45:35.188 INFO:teuthology.task.internal:Creating test directory... 2026-02-26T03:45:35.188 DEBUG:teuthology.orchestra.run.trial160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-26T03:45:35.191 DEBUG:teuthology.orchestra.run.trial195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-26T03:45:35.237 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-26T03:45:35.248 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-26T03:45:35.263 INFO:teuthology.task.internal:Creating archive directory... 2026-02-26T03:45:35.264 DEBUG:teuthology.orchestra.run.trial160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-26T03:45:35.266 DEBUG:teuthology.orchestra.run.trial195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-26T03:45:35.301 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-26T03:45:35.312 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-26T03:45:35.313 DEBUG:teuthology.orchestra.run.trial160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-26T03:45:35.324 DEBUG:teuthology.orchestra.run.trial195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-26T03:45:35.345 INFO:teuthology.orchestra.run.trial160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:45:35.354 INFO:teuthology.orchestra.run.trial160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:45:35.358 INFO:teuthology.orchestra.run.trial195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:45:35.366 INFO:teuthology.orchestra.run.trial195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:45:35.368 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-26T03:45:35.380 INFO:teuthology.task.internal:Configuring sudo... 2026-02-26T03:45:35.381 DEBUG:teuthology.orchestra.run.trial160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-26T03:45:35.398 DEBUG:teuthology.orchestra.run.trial195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-26T03:45:35.433 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-26T03:45:35.451 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-26T03:45:35.451 DEBUG:teuthology.orchestra.run.trial160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-26T03:45:35.462 DEBUG:teuthology.orchestra.run.trial195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-26T03:45:35.490 DEBUG:teuthology.orchestra.run.trial160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:45:35.632 DEBUG:teuthology.orchestra.run.trial160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:45:35.658 DEBUG:teuthology.orchestra.run.trial160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:45:35.803 DEBUG:teuthology.orchestra.run.trial160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:45:35.828 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:45:35.828 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-26T03:45:35.892 DEBUG:teuthology.orchestra.run.trial195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:45:36.003 DEBUG:teuthology.orchestra.run.trial195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:45:36.028 DEBUG:teuthology.orchestra.run.trial195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:45:36.175 DEBUG:teuthology.orchestra.run.trial195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:45:36.201 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:45:36.201 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-26T03:45:36.265 DEBUG:teuthology.orchestra.run.trial160:> sudo service rsyslog restart 2026-02-26T03:45:36.267 DEBUG:teuthology.orchestra.run.trial195:> sudo service rsyslog restart 2026-02-26T03:45:36.295 INFO:teuthology.orchestra.run.trial160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-26T03:45:36.333 INFO:teuthology.orchestra.run.trial195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-26T03:45:36.789 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-26T03:45:36.802 INFO:teuthology.task.internal:Starting timer... 2026-02-26T03:45:36.802 INFO:teuthology.run_tasks:Running task pcp... 2026-02-26T03:45:36.819 INFO:teuthology.run_tasks:Running task selinux... 2026-02-26T03:45:36.832 DEBUG:teuthology.orchestra.run.trial160:> sudo service auditd rotate 2026-02-26T03:45:36.861 INFO:teuthology.orchestra.run.trial160.stdout:Rotating logs: 2026-02-26T03:45:36.864 DEBUG:teuthology.orchestra.run.trial195:> sudo service auditd rotate 2026-02-26T03:45:36.893 INFO:teuthology.orchestra.run.trial195.stdout:Rotating logs: 2026-02-26T03:45:36.895 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-26T03:45:36.896 DEBUG:teuthology.orchestra.run.trial160:> /usr/sbin/getenforce 2026-02-26T03:45:36.920 INFO:teuthology.orchestra.run.trial160.stdout:Permissive 2026-02-26T03:45:36.920 DEBUG:teuthology.orchestra.run.trial195:> /usr/sbin/getenforce 2026-02-26T03:45:36.951 INFO:teuthology.orchestra.run.trial195.stdout:Permissive 2026-02-26T03:45:36.952 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial160.front.sepia.ceph.com': 'permissive', 'ubuntu@trial195.front.sepia.ceph.com': 'permissive'} 2026-02-26T03:45:36.952 DEBUG:teuthology.orchestra.run.trial160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-26T03:45:36.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:45:36.985 DEBUG:teuthology.orchestra.run.trial195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-26T03:45:37.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:45:37.017 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-26T03:45:37.017 DEBUG:teuthology.orchestra.run.trial160:> sudo /usr/sbin/setenforce permissive 2026-02-26T03:45:37.048 DEBUG:teuthology.orchestra.run.trial195:> sudo /usr/sbin/setenforce permissive 2026-02-26T03:45:37.081 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-26T03:45:37.098 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-26T03:45:37.112 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-26T03:45:37.113 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial160.front.sepia.ceph.com,trial195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-26T03:48:37.263 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial160.front.sepia.ceph.com'), Remote(name='ubuntu@trial195.front.sepia.ceph.com')] 2026-02-26T03:48:37.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial160.front.sepia.ceph.com' 2026-02-26T03:48:37.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:48:37.372 DEBUG:teuthology.orchestra.run.trial160:> true 2026-02-26T03:48:37.450 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial160.front.sepia.ceph.com' 2026-02-26T03:48:37.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial195.front.sepia.ceph.com' 2026-02-26T03:48:37.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T03:48:37.529 DEBUG:teuthology.orchestra.run.trial195:> true 2026-02-26T03:48:37.601 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial195.front.sepia.ceph.com' 2026-02-26T03:48:37.602 INFO:teuthology.run_tasks:Running task clock... 2026-02-26T03:48:37.616 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-26T03:48:37.616 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-26T03:48:37.617 DEBUG:teuthology.orchestra.run.trial160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-26T03:48:37.619 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-26T03:48:37.619 DEBUG:teuthology.orchestra.run.trial195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-26T03:48:37.643 INFO:teuthology.orchestra.run.trial160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-26T03:48:37.657 INFO:teuthology.orchestra.run.trial160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-26T03:48:37.669 INFO:teuthology.orchestra.run.trial195.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-26T03:48:37.678 INFO:teuthology.orchestra.run.trial160.stderr:sudo: ntpd: command not found 2026-02-26T03:48:37.680 INFO:teuthology.orchestra.run.trial195.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-26T03:48:37.688 INFO:teuthology.orchestra.run.trial160.stdout:506 Cannot talk to daemon 2026-02-26T03:48:37.700 INFO:teuthology.orchestra.run.trial160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-26T03:48:37.702 INFO:teuthology.orchestra.run.trial195.stderr:sudo: ntpd: command not found 2026-02-26T03:48:37.712 INFO:teuthology.orchestra.run.trial195.stdout:506 Cannot talk to daemon 2026-02-26T03:48:37.712 INFO:teuthology.orchestra.run.trial160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-26T03:48:37.723 INFO:teuthology.orchestra.run.trial195.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-26T03:48:37.735 INFO:teuthology.orchestra.run.trial195.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-26T03:48:37.762 INFO:teuthology.orchestra.run.trial160.stderr:bash: line 1: ntpq: command not found 2026-02-26T03:48:37.783 INFO:teuthology.orchestra.run.trial160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-26T03:48:37.783 INFO:teuthology.orchestra.run.trial160.stdout:=============================================================================== 2026-02-26T03:48:37.783 INFO:teuthology.orchestra.run.trial160.stdout:^? y.ns.gin.ntt.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:48:37.783 INFO:teuthology.orchestra.run.trial160.stdout:^? linode1.ernest-doss.org 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:48:37.783 INFO:teuthology.orchestra.run.trial160.stdout:^? rn-02.koehn.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:48:37.783 INFO:teuthology.orchestra.run.trial160.stdout:^? lax1.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:48:37.783 INFO:teuthology.orchestra.run.trial195.stderr:bash: line 1: ntpq: command not found 2026-02-26T03:48:37.807 INFO:teuthology.orchestra.run.trial195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-26T03:48:37.807 INFO:teuthology.orchestra.run.trial195.stdout:=============================================================================== 2026-02-26T03:48:37.807 INFO:teuthology.orchestra.run.trial195.stdout:^? 96-19-94-82.cpe.sparklig> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:48:37.807 INFO:teuthology.orchestra.run.trial195.stdout:^? 69.9.131.124 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:48:37.807 INFO:teuthology.orchestra.run.trial195.stdout:^? 69.176.84.38 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:48:37.807 INFO:teuthology.orchestra.run.trial195.stdout:^? 170-187-147-56.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-26T03:48:37.808 INFO:teuthology.run_tasks:Running task install... 2026-02-26T03:48:37.824 DEBUG:teuthology.task.install:project ceph 2026-02-26T03:48:37.824 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'}} 2026-02-26T03:48:37.825 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'} 2026-02-26T03:48:37.825 INFO:teuthology.task.install:Using flavor: debug 2026-02-26T03:48:37.832 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-26T03:48:37.832 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-26T03:48:37.833 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'tag': None, 'wait_for_package': False} 2026-02-26T03:48:37.833 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:48:37.836 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'tag': None, 'wait_for_package': False} 2026-02-26T03:48:37.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:48:38.028 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/ 2026-02-26T03:48:38.028 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5312.gabffb617 2026-02-26T03:48:38.037 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/ 2026-02-26T03:48:38.037 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5312.gabffb617 2026-02-26T03:48:38.331 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-26T03:48:38.331 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:48:38.331 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-26T03:48:38.358 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo, python3-pytest on remote rpm x86_64 2026-02-26T03:48:38.358 DEBUG:teuthology.orchestra.run.trial160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-26T03:48:38.382 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-26T03:48:38.382 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:48:38.382 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-26T03:48:38.409 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo, python3-pytest on remote rpm x86_64 2026-02-26T03:48:38.410 DEBUG:teuthology.orchestra.run.trial195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-26T03:48:38.424 DEBUG:teuthology.orchestra.run.trial160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-26T03:48:38.475 DEBUG:teuthology.orchestra.run.trial195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-26T03:48:38.498 DEBUG:teuthology.orchestra.run.trial160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-26T03:48:38.548 DEBUG:teuthology.orchestra.run.trial195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-26T03:48:38.562 INFO:teuthology.orchestra.run.trial160.stdout:check_obsoletes = 1 2026-02-26T03:48:38.564 DEBUG:teuthology.orchestra.run.trial160:> sudo yum clean all 2026-02-26T03:48:38.612 INFO:teuthology.orchestra.run.trial195.stdout:check_obsoletes = 1 2026-02-26T03:48:38.614 DEBUG:teuthology.orchestra.run.trial195:> sudo yum clean all 2026-02-26T03:48:38.715 INFO:teuthology.orchestra.run.trial160.stdout:41 files removed 2026-02-26T03:48:38.730 DEBUG:teuthology.orchestra.run.trial160:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-osd-crimson ceph-osd-crimson-debuginfo python3-pytest 2026-02-26T03:48:38.762 INFO:teuthology.orchestra.run.trial195.stdout:41 files removed 2026-02-26T03:48:38.777 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-osd-crimson ceph-osd-crimson-debuginfo python3-pytest 2026-02-26T03:48:39.412 INFO:teuthology.orchestra.run.trial160.stdout:ceph packages for x86_64 176 kB/s | 94 kB 00:00 2026-02-26T03:48:39.465 INFO:teuthology.orchestra.run.trial195.stdout:ceph packages for x86_64 174 kB/s | 94 kB 00:00 2026-02-26T03:48:39.887 INFO:teuthology.orchestra.run.trial160.stdout:ceph noarch packages 55 kB/s | 24 kB 00:00 2026-02-26T03:48:39.935 INFO:teuthology.orchestra.run.trial195.stdout:ceph noarch packages 55 kB/s | 24 kB 00:00 2026-02-26T03:48:40.028 INFO:teuthology.orchestra.run.trial160.stdout:ceph source packages 18 kB/s | 2.1 kB 00:00 2026-02-26T03:48:40.063 INFO:teuthology.orchestra.run.trial195.stdout:ceph source packages 20 kB/s | 2.1 kB 00:00 2026-02-26T03:48:41.481 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - BaseOS 6.4 MB/s | 8.9 MB 00:01 2026-02-26T03:48:42.004 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.9 MB 00:01 2026-02-26T03:48:45.394 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 27 MB 00:03 2026-02-26T03:48:48.901 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 8.0 MB 00:01 2026-02-26T03:48:49.770 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - Extras packages 152 kB/s | 20 kB 00:00 2026-02-26T03:48:50.508 INFO:teuthology.orchestra.run.trial195.stdout:Extra Packages for Enterprise Linux 30 MB/s | 20 MB 00:00 2026-02-26T03:48:53.718 INFO:teuthology.orchestra.run.trial195.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-26T03:48:54.960 INFO:teuthology.orchestra.run.trial195.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-26T03:48:54.960 INFO:teuthology.orchestra.run.trial195.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-26T03:48:54.988 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-26T03:48:54.993 INFO:teuthology.orchestra.run.trial195.stdout:======================================================================================= 2026-02-26T03:48:54.993 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repository Size 2026-02-26T03:48:54.993 INFO:teuthology.orchestra.run.trial195.stdout:======================================================================================= 2026-02-26T03:48:54.993 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-26T03:48:54.994 INFO:teuthology.orchestra.run.trial195.stdout: ceph x86_64 2:20.3.0-5312.gabffb617.el9 ceph 6.5 k 2026-02-26T03:48:54.994 INFO:teuthology.orchestra.run.trial195.stdout: ceph-base x86_64 2:20.3.0-5312.gabffb617.el9 ceph 5.1 M 2026-02-26T03:48:54.994 INFO:teuthology.orchestra.run.trial195.stdout: ceph-fuse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 914 k 2026-02-26T03:48:54.994 INFO:teuthology.orchestra.run.trial195.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5312.gabffb617.el9 ceph 149 k 2026-02-26T03:48:54.994 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr x86_64 2:20.3.0-5312.gabffb617.el9 ceph 937 k 2026-02-26T03:48:54.994 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 184 k 2026-02-26T03:48:54.994 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 51 M 2026-02-26T03:48:54.994 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 7.4 M 2026-02-26T03:48:54.994 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-rook noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 50 k 2026-02-26T03:48:54.994 INFO:teuthology.orchestra.run.trial195.stdout: ceph-osd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 163 k 2026-02-26T03:48:54.994 INFO:teuthology.orchestra.run.trial195.stdout: ceph-osd-classic x86_64 2:20.3.0-5312.gabffb617.el9 ceph 15 M 2026-02-26T03:48:54.995 INFO:teuthology.orchestra.run.trial195.stdout: ceph-osd-crimson x86_64 2:20.3.0-5312.gabffb617.el9 ceph 87 M 2026-02-26T03:48:54.995 INFO:teuthology.orchestra.run.trial195.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5312.gabffb617.el9 ceph 429 M 2026-02-26T03:48:54.995 INFO:teuthology.orchestra.run.trial195.stdout: ceph-radosgw x86_64 2:20.3.0-5312.gabffb617.el9 ceph 11 M 2026-02-26T03:48:54.995 INFO:teuthology.orchestra.run.trial195.stdout: ceph-test x86_64 2:20.3.0-5312.gabffb617.el9 ceph 126 M 2026-02-26T03:48:54.995 INFO:teuthology.orchestra.run.trial195.stdout: ceph-volume noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 301 k 2026-02-26T03:48:54.995 INFO:teuthology.orchestra.run.trial195.stdout: cephadm noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 1.0 M 2026-02-26T03:48:54.995 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs-devel x86_64 2:20.3.0-5312.gabffb617.el9 ceph 37 k 2026-02-26T03:48:54.995 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 859 k 2026-02-26T03:48:54.995 INFO:teuthology.orchestra.run.trial195.stdout: librados-devel x86_64 2:20.3.0-5312.gabffb617.el9 ceph 124 k 2026-02-26T03:48:54.995 INFO:teuthology.orchestra.run.trial195.stdout: python3-cephfs x86_64 2:20.3.0-5312.gabffb617.el9 ceph 203 k 2026-02-26T03:48:54.995 INFO:teuthology.orchestra.run.trial195.stdout: python3-pytest noarch 6.2.2-7.el9 appstream 519 k 2026-02-26T03:48:54.996 INFO:teuthology.orchestra.run.trial195.stdout: python3-rados x86_64 2:20.3.0-5312.gabffb617.el9 ceph 324 k 2026-02-26T03:48:54.996 INFO:teuthology.orchestra.run.trial195.stdout: python3-rbd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 305 k 2026-02-26T03:48:54.996 INFO:teuthology.orchestra.run.trial195.stdout: python3-rgw x86_64 2:20.3.0-5312.gabffb617.el9 ceph 100 k 2026-02-26T03:48:54.996 INFO:teuthology.orchestra.run.trial195.stdout: rbd-fuse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 87 k 2026-02-26T03:48:54.996 INFO:teuthology.orchestra.run.trial195.stdout: rbd-mirror x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.6 M 2026-02-26T03:48:54.996 INFO:teuthology.orchestra.run.trial195.stdout: rbd-nbd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 166 k 2026-02-26T03:48:54.996 INFO:teuthology.orchestra.run.trial195.stdout:Upgrading: 2026-02-26T03:48:54.996 INFO:teuthology.orchestra.run.trial195.stdout: librados2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 3.3 M 2026-02-26T03:48:54.996 INFO:teuthology.orchestra.run.trial195.stdout: librbd1 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.7 M 2026-02-26T03:48:54.996 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-26T03:48:54.996 INFO:teuthology.orchestra.run.trial195.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-26T03:48:54.997 INFO:teuthology.orchestra.run.trial195.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-26T03:48:54.997 INFO:teuthology.orchestra.run.trial195.stdout: ceph-common x86_64 2:20.3.0-5312.gabffb617.el9 ceph 21 M 2026-02-26T03:48:54.997 INFO:teuthology.orchestra.run.trial195.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 57 k 2026-02-26T03:48:54.997 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mds x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.0 M 2026-02-26T03:48:54.997 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 297 k 2026-02-26T03:48:54.997 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mon x86_64 2:20.3.0-5312.gabffb617.el9 ceph 4.9 M 2026-02-26T03:48:54.997 INFO:teuthology.orchestra.run.trial195.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 17 k 2026-02-26T03:48:54.997 INFO:teuthology.orchestra.run.trial195.stdout: ceph-selinux x86_64 2:20.3.0-5312.gabffb617.el9 ceph 25 k 2026-02-26T03:48:54.997 INFO:teuthology.orchestra.run.trial195.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-26T03:48:54.997 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-26T03:48:54.997 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 26 k 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: libcephsqlite x86_64 2:20.3.0-5312.gabffb617.el9 ceph 153 k 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-26T03:48:54.998 INFO:teuthology.orchestra.run.trial195.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-26T03:48:54.999 INFO:teuthology.orchestra.run.trial195.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-26T03:48:54.999 INFO:teuthology.orchestra.run.trial195.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-26T03:48:54.999 INFO:teuthology.orchestra.run.trial195.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-26T03:48:54.999 INFO:teuthology.orchestra.run.trial195.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-26T03:48:54.999 INFO:teuthology.orchestra.run.trial195.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-26T03:48:54.999 INFO:teuthology.orchestra.run.trial195.stdout: libradosstriper1 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 263 k 2026-02-26T03:48:54.999 INFO:teuthology.orchestra.run.trial195.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-26T03:48:54.999 INFO:teuthology.orchestra.run.trial195.stdout: librgw2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 5.6 M 2026-02-26T03:48:54.999 INFO:teuthology.orchestra.run.trial195.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-26T03:48:54.999 INFO:teuthology.orchestra.run.trial195.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-26T03:48:54.999 INFO:teuthology.orchestra.run.trial195.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-26T03:48:55.000 INFO:teuthology.orchestra.run.trial195.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-ceph-argparse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 45 k 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-ceph-common x86_64 2:20.3.0-5312.gabffb617.el9 ceph 185 k 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-26T03:48:55.001 INFO:teuthology.orchestra.run.trial195.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-iniconfig noarch 1.1.1-7.el9 appstream 17 k 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-26T03:48:55.002 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-26T03:48:55.003 INFO:teuthology.orchestra.run.trial195.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-pluggy noarch 0.13.1-7.el9 appstream 41 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-py noarch 1.10.0-6.el9 appstream 477 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-26T03:48:55.004 INFO:teuthology.orchestra.run.trial195.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-26T03:48:55.005 INFO:teuthology.orchestra.run.trial195.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout:Installing weak dependencies: 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout: ceph-debugsource x86_64 2:20.3.0-5312.gabffb617.el9 ceph 12 M 2026-02-26T03:48:55.006 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 22 k 2026-02-26T03:48:55.007 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs-daemon x86_64 2:20.3.0-5312.gabffb617.el9 ceph 37 k 2026-02-26T03:48:55.007 INFO:teuthology.orchestra.run.trial195.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-26T03:48:55.007 INFO:teuthology.orchestra.run.trial195.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-26T03:48:55.007 INFO:teuthology.orchestra.run.trial195.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-26T03:48:55.007 INFO:teuthology.orchestra.run.trial195.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-26T03:48:55.007 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:48:55.007 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-26T03:48:55.007 INFO:teuthology.orchestra.run.trial195.stdout:======================================================================================= 2026-02-26T03:48:55.007 INFO:teuthology.orchestra.run.trial195.stdout:Install 156 Packages 2026-02-26T03:48:55.007 INFO:teuthology.orchestra.run.trial195.stdout:Upgrade 2 Packages 2026-02-26T03:48:55.007 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:48:55.008 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 863 M 2026-02-26T03:48:55.008 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-26T03:48:55.355 INFO:teuthology.orchestra.run.trial195.stdout:(1/158): ceph-20.3.0-5312.gabffb617.el9.x86_64. 128 kB/s | 6.5 kB 00:00 2026-02-26T03:48:55.594 INFO:teuthology.orchestra.run.trial195.stdout:(2/158): ceph-base-20.3.0-5312.gabffb617.el9.x8 18 MB/s | 5.1 MB 00:00 2026-02-26T03:48:55.647 INFO:teuthology.orchestra.run.trial195.stdout:(3/158): ceph-fuse-20.3.0-5312.gabffb617.el9.x8 17 MB/s | 914 kB 00:00 2026-02-26T03:48:55.686 INFO:teuthology.orchestra.run.trial195.stdout:(4/158): ceph-immutable-object-cache-20.3.0-531 3.7 MB/s | 149 kB 00:00 2026-02-26T03:48:55.803 INFO:teuthology.orchestra.run.trial195.stdout:(5/158): ceph-mds-20.3.0-5312.gabffb617.el9.x86 17 MB/s | 2.0 MB 00:00 2026-02-26T03:48:55.864 INFO:teuthology.orchestra.run.trial195.stdout:(6/158): ceph-mgr-20.3.0-5312.gabffb617.el9.x86 15 MB/s | 937 kB 00:00 2026-02-26T03:48:55.969 INFO:teuthology.orchestra.run.trial195.stdout:(7/158): ceph-debugsource-20.3.0-5312.gabffb617 20 MB/s | 12 MB 00:00 2026-02-26T03:48:56.027 INFO:teuthology.orchestra.run.trial195.stdout:(8/158): ceph-osd-20.3.0-5312.gabffb617.el9.x86 2.8 MB/s | 163 kB 00:00 2026-02-26T03:48:56.290 INFO:teuthology.orchestra.run.trial195.stdout:(9/158): ceph-common-20.3.0-5312.gabffb617.el9. 21 MB/s | 21 MB 00:00 2026-02-26T03:48:56.297 INFO:teuthology.orchestra.run.trial195.stdout:(10/158): ceph-mon-20.3.0-5312.gabffb617.el9.x8 11 MB/s | 4.9 MB 00:00 2026-02-26T03:48:56.978 INFO:teuthology.orchestra.run.trial195.stdout:(11/158): ceph-osd-classic-20.3.0-5312.gabffb61 16 MB/s | 15 MB 00:00 2026-02-26T03:48:57.911 INFO:teuthology.orchestra.run.trial195.stdout:(12/158): ceph-radosgw-20.3.0-5312.gabffb617.el 12 MB/s | 11 MB 00:00 2026-02-26T03:48:57.955 INFO:teuthology.orchestra.run.trial195.stdout:(13/158): ceph-selinux-20.3.0-5312.gabffb617.el 582 kB/s | 25 kB 00:00 2026-02-26T03:49:04.241 INFO:teuthology.orchestra.run.trial195.stdout:(14/158): ceph-osd-crimson-20.3.0-5312.gabffb61 11 MB/s | 87 MB 00:07 2026-02-26T03:49:04.262 INFO:teuthology.orchestra.run.trial195.stdout:(15/158): libcephfs-daemon-20.3.0-5312.gabffb61 1.7 MB/s | 37 kB 00:00 2026-02-26T03:49:04.318 INFO:teuthology.orchestra.run.trial195.stdout:(16/158): libcephfs-devel-20.3.0-5312.gabffb617 668 kB/s | 37 kB 00:00 2026-02-26T03:49:04.374 INFO:teuthology.orchestra.run.trial195.stdout:(17/158): libcephfs-proxy2-20.3.0-5312.gabffb61 457 kB/s | 26 kB 00:00 2026-02-26T03:49:04.501 INFO:teuthology.orchestra.run.trial195.stdout:(18/158): libcephfs2-20.3.0-5312.gabffb617.el9. 6.6 MB/s | 859 kB 00:00 2026-02-26T03:49:04.560 INFO:teuthology.orchestra.run.trial195.stdout:(19/158): libcephsqlite-20.3.0-5312.gabffb617.e 2.6 MB/s | 153 kB 00:00 2026-02-26T03:49:04.617 INFO:teuthology.orchestra.run.trial195.stdout:(20/158): librados-devel-20.3.0-5312.gabffb617. 2.1 MB/s | 124 kB 00:00 2026-02-26T03:49:04.678 INFO:teuthology.orchestra.run.trial195.stdout:(21/158): libradosstriper1-20.3.0-5312.gabffb61 4.2 MB/s | 263 kB 00:00 2026-02-26T03:49:05.394 INFO:teuthology.orchestra.run.trial195.stdout:(22/158): librgw2-20.3.0-5312.gabffb617.el9.x86 7.9 MB/s | 5.6 MB 00:00 2026-02-26T03:49:05.450 INFO:teuthology.orchestra.run.trial195.stdout:(23/158): python3-ceph-argparse-20.3.0-5312.gab 800 kB/s | 45 kB 00:00 2026-02-26T03:49:05.514 INFO:teuthology.orchestra.run.trial195.stdout:(24/158): python3-ceph-common-20.3.0-5312.gabff 2.9 MB/s | 185 kB 00:00 2026-02-26T03:49:05.581 INFO:teuthology.orchestra.run.trial195.stdout:(25/158): python3-cephfs-20.3.0-5312.gabffb617. 2.9 MB/s | 203 kB 00:00 2026-02-26T03:49:06.048 INFO:teuthology.orchestra.run.trial195.stdout:(26/158): python3-rados-20.3.0-5312.gabffb617.e 693 kB/s | 324 kB 00:00 2026-02-26T03:49:06.105 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 27 MB 00:23 2026-02-26T03:49:06.112 INFO:teuthology.orchestra.run.trial195.stdout:(27/158): python3-rbd-20.3.0-5312.gabffb617.el9 4.7 MB/s | 305 kB 00:00 2026-02-26T03:49:06.172 INFO:teuthology.orchestra.run.trial195.stdout:(28/158): python3-rgw-20.3.0-5312.gabffb617.el9 1.6 MB/s | 100 kB 00:00 2026-02-26T03:49:06.233 INFO:teuthology.orchestra.run.trial195.stdout:(29/158): rbd-fuse-20.3.0-5312.gabffb617.el9.x8 1.4 MB/s | 87 kB 00:00 2026-02-26T03:49:06.692 INFO:teuthology.orchestra.run.trial195.stdout:(30/158): rbd-mirror-20.3.0-5312.gabffb617.el9. 5.6 MB/s | 2.6 MB 00:00 2026-02-26T03:49:06.760 INFO:teuthology.orchestra.run.trial195.stdout:(31/158): rbd-nbd-20.3.0-5312.gabffb617.el9.x86 2.4 MB/s | 166 kB 00:00 2026-02-26T03:49:06.826 INFO:teuthology.orchestra.run.trial195.stdout:(32/158): ceph-grafana-dashboards-20.3.0-5312.g 866 kB/s | 57 kB 00:00 2026-02-26T03:49:06.896 INFO:teuthology.orchestra.run.trial195.stdout:(33/158): ceph-mgr-cephadm-20.3.0-5312.gabffb61 2.6 MB/s | 184 kB 00:00 2026-02-26T03:49:07.507 INFO:teuthology.orchestra.run.trial195.stdout:(34/158): ceph-test-20.3.0-5312.gabffb617.el9.x 13 MB/s | 126 MB 00:09 2026-02-26T03:49:08.170 INFO:teuthology.orchestra.run.trial195.stdout:(35/158): ceph-mgr-diskprediction-local-20.3.0- 11 MB/s | 7.4 MB 00:00 2026-02-26T03:49:08.224 INFO:teuthology.orchestra.run.trial195.stdout:(36/158): ceph-mgr-k8sevents-20.3.0-5312.gabffb 419 kB/s | 22 kB 00:00 2026-02-26T03:49:08.282 INFO:teuthology.orchestra.run.trial195.stdout:(37/158): ceph-mgr-modules-core-20.3.0-5312.gab 5.0 MB/s | 297 kB 00:00 2026-02-26T03:49:08.335 INFO:teuthology.orchestra.run.trial195.stdout:(38/158): ceph-mgr-rook-20.3.0-5312.gabffb617.e 947 kB/s | 50 kB 00:00 2026-02-26T03:49:08.387 INFO:teuthology.orchestra.run.trial195.stdout:(39/158): ceph-prometheus-alerts-20.3.0-5312.ga 331 kB/s | 17 kB 00:00 2026-02-26T03:49:08.446 INFO:teuthology.orchestra.run.trial195.stdout:(40/158): ceph-volume-20.3.0-5312.gabffb617.el9 5.1 MB/s | 301 kB 00:00 2026-02-26T03:49:08.570 INFO:teuthology.orchestra.run.trial195.stdout:(41/158): cephadm-20.3.0-5312.gabffb617.el9.noa 8.2 MB/s | 1.0 MB 00:00 2026-02-26T03:49:08.627 INFO:teuthology.orchestra.run.trial195.stdout:(42/158): cryptsetup-2.8.1-3.el9.x86_64.rpm 6.0 MB/s | 351 kB 00:00 2026-02-26T03:49:08.662 INFO:teuthology.orchestra.run.trial195.stdout:(43/158): hwloc-libs-2.4.1-6.el9.x86_64.rpm 60 MB/s | 2.1 MB 00:00 2026-02-26T03:49:08.671 INFO:teuthology.orchestra.run.trial195.stdout:(44/158): ledmon-libs-1.1.0-3.el9.x86_64.rpm 4.4 MB/s | 40 kB 00:00 2026-02-26T03:49:08.680 INFO:teuthology.orchestra.run.trial195.stdout:(45/158): libatomic-11.5.0-14.el9.x86_64.rpm 2.6 MB/s | 23 kB 00:00 2026-02-26T03:49:08.689 INFO:teuthology.orchestra.run.trial195.stdout:(46/158): libconfig-1.7.2-9.el9.x86_64.rpm 7.6 MB/s | 72 kB 00:00 2026-02-26T03:49:08.701 INFO:teuthology.orchestra.run.trial195.stdout:(47/158): libgfortran-11.5.0-14.el9.x86_64.rpm 64 MB/s | 794 kB 00:00 2026-02-26T03:49:08.712 INFO:teuthology.orchestra.run.trial195.stdout:(48/158): libquadmath-11.5.0-14.el9.x86_64.rpm 17 MB/s | 184 kB 00:00 2026-02-26T03:49:08.722 INFO:teuthology.orchestra.run.trial195.stdout:(49/158): lksctp-tools-1.0.19-2.el9.x86_64.rpm 9.5 MB/s | 94 kB 00:00 2026-02-26T03:49:08.731 INFO:teuthology.orchestra.run.trial195.stdout:(50/158): mailcap-2.1.49-5.el9.noarch.rpm 3.7 MB/s | 33 kB 00:00 2026-02-26T03:49:08.743 INFO:teuthology.orchestra.run.trial195.stdout:(51/158): nvme-cli-2.16-1.el9.x86_64.rpm 96 MB/s | 1.2 MB 00:00 2026-02-26T03:49:08.753 INFO:teuthology.orchestra.run.trial195.stdout:(52/158): pciutils-3.7.0-7.el9.x86_64.rpm 9.3 MB/s | 93 kB 00:00 2026-02-26T03:49:08.767 INFO:teuthology.orchestra.run.trial195.stdout:(53/158): python3-cffi-1.14.5-5.el9.x86_64.rpm 18 MB/s | 253 kB 00:00 2026-02-26T03:49:08.787 INFO:teuthology.orchestra.run.trial195.stdout:(54/158): python3-cryptography-36.0.1-5.el9.x86 63 MB/s | 1.2 MB 00:00 2026-02-26T03:49:08.799 INFO:teuthology.orchestra.run.trial195.stdout:(55/158): python3-ply-3.11-14.el9.noarch.rpm 8.2 MB/s | 106 kB 00:00 2026-02-26T03:49:08.811 INFO:teuthology.orchestra.run.trial195.stdout:(56/158): python3-pycparser-2.20-6.el9.noarch.r 11 MB/s | 135 kB 00:00 2026-02-26T03:49:08.823 INFO:teuthology.orchestra.run.trial195.stdout:(57/158): python3-pyparsing-2.4.7-9.el9.noarch. 13 MB/s | 150 kB 00:00 2026-02-26T03:49:08.834 INFO:teuthology.orchestra.run.trial195.stdout:(58/158): python3-requests-2.25.1-10.el9.noarch 11 MB/s | 126 kB 00:00 2026-02-26T03:49:08.847 INFO:teuthology.orchestra.run.trial195.stdout:(59/158): python3-urllib3-1.26.5-7.el9.noarch.r 17 MB/s | 218 kB 00:00 2026-02-26T03:49:08.858 INFO:teuthology.orchestra.run.trial195.stdout:(60/158): unzip-6.0-59.el9.x86_64.rpm 15 MB/s | 182 kB 00:00 2026-02-26T03:49:08.869 INFO:teuthology.orchestra.run.trial195.stdout:(61/158): zip-3.0-35.el9.x86_64.rpm 24 MB/s | 266 kB 00:00 2026-02-26T03:49:08.938 INFO:teuthology.orchestra.run.trial195.stdout:(62/158): boost-program-options-1.75.0-13.el9.x 1.5 MB/s | 104 kB 00:00 2026-02-26T03:49:08.960 INFO:teuthology.orchestra.run.trial195.stdout:(63/158): flexiblas-3.0.4-9.el9.x86_64.rpm 1.3 MB/s | 30 kB 00:00 2026-02-26T03:49:09.069 INFO:teuthology.orchestra.run.trial195.stdout:(64/158): flexiblas-netlib-3.0.4-9.el9.x86_64.r 27 MB/s | 3.0 MB 00:00 2026-02-26T03:49:09.091 INFO:teuthology.orchestra.run.trial195.stdout:(65/158): flexiblas-openblas-openmp-3.0.4-9.el9 685 kB/s | 15 kB 00:00 2026-02-26T03:49:09.115 INFO:teuthology.orchestra.run.trial195.stdout:(66/158): libasan8-15.2.1-7.el9.x86_64.rpm 21 MB/s | 525 kB 00:00 2026-02-26T03:49:09.139 INFO:teuthology.orchestra.run.trial195.stdout:(67/158): libnbd-1.20.3-4.el9.x86_64.rpm 6.8 MB/s | 164 kB 00:00 2026-02-26T03:49:09.162 INFO:teuthology.orchestra.run.trial195.stdout:(68/158): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.7 MB/s | 160 kB 00:00 2026-02-26T03:49:09.184 INFO:teuthology.orchestra.run.trial195.stdout:(69/158): librabbitmq-0.11.0-7.el9.x86_64.rpm 2.0 MB/s | 45 kB 00:00 2026-02-26T03:49:09.208 INFO:teuthology.orchestra.run.trial195.stdout:(70/158): librdkafka-1.6.1-102.el9.x86_64.rpm 27 MB/s | 662 kB 00:00 2026-02-26T03:49:09.232 INFO:teuthology.orchestra.run.trial195.stdout:(71/158): libstoragemgmt-1.10.1-1.el9.x86_64.rp 10 MB/s | 246 kB 00:00 2026-02-26T03:49:09.256 INFO:teuthology.orchestra.run.trial195.stdout:(72/158): libubsan-11.5.0-14.el9.x86_64.rpm 7.3 MB/s | 176 kB 00:00 2026-02-26T03:49:09.280 INFO:teuthology.orchestra.run.trial195.stdout:(73/158): libxslt-1.1.34-12.el9.x86_64.rpm 9.6 MB/s | 233 kB 00:00 2026-02-26T03:49:09.304 INFO:teuthology.orchestra.run.trial195.stdout:(74/158): lttng-ust-2.12.0-6.el9.x86_64.rpm 12 MB/s | 292 kB 00:00 2026-02-26T03:49:09.327 INFO:teuthology.orchestra.run.trial195.stdout:(75/158): lua-5.4.4-4.el9.x86_64.rpm 7.9 MB/s | 188 kB 00:00 2026-02-26T03:49:09.349 INFO:teuthology.orchestra.run.trial195.stdout:(76/158): openblas-0.3.29-1.el9.x86_64.rpm 1.9 MB/s | 42 kB 00:00 2026-02-26T03:49:09.439 INFO:teuthology.orchestra.run.trial195.stdout:(77/158): openblas-openmp-0.3.29-1.el9.x86_64.r 58 MB/s | 5.3 MB 00:00 2026-02-26T03:49:09.465 INFO:teuthology.orchestra.run.trial195.stdout:(78/158): protobuf-3.14.0-17.el9.x86_64.rpm 39 MB/s | 1.0 MB 00:00 2026-02-26T03:49:09.560 INFO:teuthology.orchestra.run.trial195.stdout:(79/158): python3-babel-2.9.1-2.el9.noarch.rpm 63 MB/s | 6.0 MB 00:00 2026-02-26T03:49:09.585 INFO:teuthology.orchestra.run.trial195.stdout:(80/158): python3-devel-3.9.25-3.el9.x86_64.rpm 9.6 MB/s | 244 kB 00:00 2026-02-26T03:49:09.607 INFO:teuthology.orchestra.run.trial195.stdout:(81/158): python3-iniconfig-1.1.1-7.el9.noarch. 791 kB/s | 17 kB 00:00 2026-02-26T03:49:09.631 INFO:teuthology.orchestra.run.trial195.stdout:(82/158): python3-jinja2-2.11.3-8.el9.noarch.rp 10 MB/s | 249 kB 00:00 2026-02-26T03:49:09.654 INFO:teuthology.orchestra.run.trial195.stdout:(83/158): python3-jmespath-1.0.1-1.el9.noarch.r 2.1 MB/s | 48 kB 00:00 2026-02-26T03:49:09.678 INFO:teuthology.orchestra.run.trial195.stdout:(84/158): python3-libstoragemgmt-1.10.1-1.el9.x 7.2 MB/s | 177 kB 00:00 2026-02-26T03:49:09.705 INFO:teuthology.orchestra.run.trial195.stdout:(85/158): python3-lxml-4.6.5-3.el9.x86_64.rpm 45 MB/s | 1.2 MB 00:00 2026-02-26T03:49:09.727 INFO:teuthology.orchestra.run.trial195.stdout:(86/158): python3-markupsafe-1.1.1-12.el9.x86_6 1.5 MB/s | 35 kB 00:00 2026-02-26T03:49:09.822 INFO:teuthology.orchestra.run.trial195.stdout:(87/158): python3-numpy-1.23.5-2.el9.x86_64.rpm 65 MB/s | 6.1 MB 00:00 2026-02-26T03:49:09.847 INFO:teuthology.orchestra.run.trial195.stdout:(88/158): python3-numpy-f2py-1.23.5-2.el9.x86_6 17 MB/s | 442 kB 00:00 2026-02-26T03:49:09.870 INFO:teuthology.orchestra.run.trial195.stdout:(89/158): python3-packaging-20.9-5.el9.noarch.r 3.3 MB/s | 77 kB 00:00 2026-02-26T03:49:09.892 INFO:teuthology.orchestra.run.trial195.stdout:(90/158): python3-pluggy-0.13.1-7.el9.noarch.rp 1.8 MB/s | 41 kB 00:00 2026-02-26T03:49:09.915 INFO:teuthology.orchestra.run.trial195.stdout:(91/158): python3-protobuf-3.14.0-17.el9.noarch 12 MB/s | 267 kB 00:00 2026-02-26T03:49:09.941 INFO:teuthology.orchestra.run.trial195.stdout:(92/158): python3-py-1.10.0-6.el9.noarch.rpm 18 MB/s | 477 kB 00:00 2026-02-26T03:49:09.965 INFO:teuthology.orchestra.run.trial195.stdout:(93/158): python3-pyasn1-0.4.8-7.el9.noarch.rpm 6.3 MB/s | 157 kB 00:00 2026-02-26T03:49:09.990 INFO:teuthology.orchestra.run.trial195.stdout:(94/158): python3-pyasn1-modules-0.4.8-7.el9.no 11 MB/s | 277 kB 00:00 2026-02-26T03:49:10.022 INFO:teuthology.orchestra.run.trial195.stdout:(95/158): python3-pytest-6.2.2-7.el9.noarch.rpm 16 MB/s | 519 kB 00:00 2026-02-26T03:49:10.045 INFO:teuthology.orchestra.run.trial195.stdout:(96/158): python3-requests-oauthlib-1.3.0-12.el 2.3 MB/s | 54 kB 00:00 2026-02-26T03:49:10.323 INFO:teuthology.orchestra.run.trial195.stdout:(97/158): python3-scipy-1.9.3-2.el9.x86_64.rpm 69 MB/s | 19 MB 00:00 2026-02-26T03:49:10.346 INFO:teuthology.orchestra.run.trial195.stdout:(98/158): python3-toml-0.10.2-6.el9.noarch.rpm 1.8 MB/s | 42 kB 00:00 2026-02-26T03:49:10.369 INFO:teuthology.orchestra.run.trial195.stdout:(99/158): qatlib-25.08.0-2.el9.x86_64.rpm 10 MB/s | 240 kB 00:00 2026-02-26T03:49:10.391 INFO:teuthology.orchestra.run.trial195.stdout:(100/158): qatlib-service-25.08.0-2.el9.x86_64. 1.7 MB/s | 37 kB 00:00 2026-02-26T03:49:10.413 INFO:teuthology.orchestra.run.trial195.stdout:(101/158): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.9 MB/s | 66 kB 00:00 2026-02-26T03:49:10.436 INFO:teuthology.orchestra.run.trial195.stdout:(102/158): socat-1.7.4.1-8.el9.x86_64.rpm 13 MB/s | 303 kB 00:00 2026-02-26T03:49:10.460 INFO:teuthology.orchestra.run.trial195.stdout:(103/158): xmlsec1-1.2.29-13.el9.x86_64.rpm 7.8 MB/s | 189 kB 00:00 2026-02-26T03:49:10.482 INFO:teuthology.orchestra.run.trial195.stdout:(104/158): xmlsec1-openssl-1.2.29-13.el9.x86_64 3.9 MB/s | 90 kB 00:00 2026-02-26T03:49:10.504 INFO:teuthology.orchestra.run.trial195.stdout:(105/158): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.8 MB/s | 64 kB 00:00 2026-02-26T03:49:10.520 INFO:teuthology.orchestra.run.trial195.stdout:(106/158): lua-devel-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 22 kB 00:00 2026-02-26T03:49:10.565 INFO:teuthology.orchestra.run.trial195.stdout:(107/158): protobuf-compiler-3.14.0-17.el9.x86_ 19 MB/s | 862 kB 00:00 2026-02-26T03:49:10.840 INFO:teuthology.orchestra.run.trial195.stdout:(108/158): abseil-cpp-20211102.0-4.el9.x86_64.r 2.0 MB/s | 551 kB 00:00 2026-02-26T03:49:10.885 INFO:teuthology.orchestra.run.trial195.stdout:(109/158): gperftools-libs-2.9.1-3.el9.x86_64.r 6.6 MB/s | 308 kB 00:00 2026-02-26T03:49:10.922 INFO:teuthology.orchestra.run.trial195.stdout:(110/158): grpc-data-1.46.7-10.el9.noarch.rpm 538 kB/s | 19 kB 00:00 2026-02-26T03:49:11.077 INFO:teuthology.orchestra.run.trial195.stdout:(111/158): libarrow-9.0.0-15.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2026-02-26T03:49:11.112 INFO:teuthology.orchestra.run.trial195.stdout:(112/158): libarrow-doc-9.0.0-15.el9.noarch.rpm 711 kB/s | 25 kB 00:00 2026-02-26T03:49:11.147 INFO:teuthology.orchestra.run.trial195.stdout:(113/158): liboath-2.6.12-1.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2026-02-26T03:49:11.185 INFO:teuthology.orchestra.run.trial195.stdout:(114/158): libunwind-1.6.2-1.el9.x86_64.rpm 1.8 MB/s | 67 kB 00:00 2026-02-26T03:49:11.224 INFO:teuthology.orchestra.run.trial195.stdout:(115/158): luarocks-3.9.2-5.el9.noarch.rpm 3.8 MB/s | 151 kB 00:00 2026-02-26T03:49:11.276 INFO:teuthology.orchestra.run.trial195.stdout:(116/158): parquet-libs-9.0.0-15.el9.x86_64.rpm 16 MB/s | 838 kB 00:00 2026-02-26T03:49:11.317 INFO:teuthology.orchestra.run.trial195.stdout:(117/158): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2026-02-26T03:49:11.353 INFO:teuthology.orchestra.run.trial195.stdout:(118/158): python3-autocommand-2.2.2-8.el9.noar 829 kB/s | 29 kB 00:00 2026-02-26T03:49:11.387 INFO:teuthology.orchestra.run.trial195.stdout:(119/158): python3-backports-tarfile-1.2.0-1.el 1.7 MB/s | 60 kB 00:00 2026-02-26T03:49:11.422 INFO:teuthology.orchestra.run.trial195.stdout:(120/158): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.2 MB/s | 43 kB 00:00 2026-02-26T03:49:11.456 INFO:teuthology.orchestra.run.trial195.stdout:(121/158): python3-cachetools-4.2.4-1.el9.noarc 947 kB/s | 32 kB 00:00 2026-02-26T03:49:11.493 INFO:teuthology.orchestra.run.trial195.stdout:(122/158): python3-certifi-2023.05.07-4.el9.noa 391 kB/s | 14 kB 00:00 2026-02-26T03:49:11.530 INFO:teuthology.orchestra.run.trial195.stdout:(123/158): python3-cheroot-10.0.1-4.el9.noarch. 4.5 MB/s | 173 kB 00:00 2026-02-26T03:49:11.573 INFO:teuthology.orchestra.run.trial195.stdout:(124/158): python3-cherrypy-18.6.1-2.el9.noarch 8.3 MB/s | 358 kB 00:00 2026-02-26T03:49:11.610 INFO:teuthology.orchestra.run.trial195.stdout:(125/158): python3-google-auth-2.45.0-1.el9.noa 6.7 MB/s | 254 kB 00:00 2026-02-26T03:49:11.689 INFO:teuthology.orchestra.run.trial195.stdout:(126/158): python3-grpcio-1.46.7-10.el9.x86_64. 26 MB/s | 2.0 MB 00:00 2026-02-26T03:49:11.728 INFO:teuthology.orchestra.run.trial195.stdout:(127/158): python3-grpcio-tools-1.46.7-10.el9.x 3.6 MB/s | 144 kB 00:00 2026-02-26T03:49:11.764 INFO:teuthology.orchestra.run.trial195.stdout:(128/158): python3-influxdb-5.3.1-1.el9.noarch. 3.7 MB/s | 139 kB 00:00 2026-02-26T03:49:11.799 INFO:teuthology.orchestra.run.trial195.stdout:(129/158): python3-isodate-0.6.1-3.el9.noarch.r 1.6 MB/s | 56 kB 00:00 2026-02-26T03:49:11.833 INFO:teuthology.orchestra.run.trial195.stdout:(130/158): python3-jaraco-8.2.1-3.el9.noarch.rp 320 kB/s | 11 kB 00:00 2026-02-26T03:49:11.866 INFO:teuthology.orchestra.run.trial195.stdout:(131/158): python3-jaraco-classes-3.2.1-5.el9.n 528 kB/s | 18 kB 00:00 2026-02-26T03:49:11.900 INFO:teuthology.orchestra.run.trial195.stdout:(132/158): python3-jaraco-collections-3.0.0-8.e 689 kB/s | 23 kB 00:00 2026-02-26T03:49:11.934 INFO:teuthology.orchestra.run.trial195.stdout:(133/158): python3-jaraco-context-6.0.1-3.el9.n 584 kB/s | 20 kB 00:00 2026-02-26T03:49:11.967 INFO:teuthology.orchestra.run.trial195.stdout:(134/158): python3-jaraco-functools-3.5.0-2.el9 580 kB/s | 19 kB 00:00 2026-02-26T03:49:12.001 INFO:teuthology.orchestra.run.trial195.stdout:(135/158): python3-jaraco-text-4.0.0-2.el9.noar 783 kB/s | 26 kB 00:00 2026-02-26T03:49:12.042 INFO:teuthology.orchestra.run.trial195.stdout:(136/158): python3-kubernetes-26.1.0-3.el9.noar 25 MB/s | 1.0 MB 00:00 2026-02-26T03:49:12.076 INFO:teuthology.orchestra.run.trial195.stdout:(137/158): python3-more-itertools-8.12.0-2.el9. 2.2 MB/s | 79 kB 00:00 2026-02-26T03:49:12.111 INFO:teuthology.orchestra.run.trial195.stdout:(138/158): python3-msgpack-1.0.3-2.el9.x86_64.r 2.4 MB/s | 86 kB 00:00 2026-02-26T03:49:12.146 INFO:teuthology.orchestra.run.trial195.stdout:(139/158): python3-natsort-7.1.1-5.el9.noarch.r 1.6 MB/s | 58 kB 00:00 2026-02-26T03:49:12.179 INFO:teuthology.orchestra.run.trial195.stdout:(140/158): python3-portend-3.1.0-2.el9.noarch.r 491 kB/s | 16 kB 00:00 2026-02-26T03:49:12.213 INFO:teuthology.orchestra.run.trial195.stdout:(141/158): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.6 MB/s | 90 kB 00:00 2026-02-26T03:49:12.247 INFO:teuthology.orchestra.run.trial195.stdout:(142/158): python3-repoze-lru-0.7-16.el9.noarch 905 kB/s | 31 kB 00:00 2026-02-26T03:49:12.283 INFO:teuthology.orchestra.run.trial195.stdout:(143/158): python3-routes-2.5.1-5.el9.noarch.rp 5.2 MB/s | 188 kB 00:00 2026-02-26T03:49:12.318 INFO:teuthology.orchestra.run.trial195.stdout:(144/158): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2026-02-26T03:49:12.353 INFO:teuthology.orchestra.run.trial195.stdout:(145/158): python3-saml-1.16.0-1.el9.noarch.rpm 3.5 MB/s | 125 kB 00:00 2026-02-26T03:49:12.387 INFO:teuthology.orchestra.run.trial195.stdout:(146/158): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2026-02-26T03:49:12.422 INFO:teuthology.orchestra.run.trial195.stdout:(147/158): python3-typing-extensions-4.15.0-1.e 2.4 MB/s | 86 kB 00:00 2026-02-26T03:49:12.457 INFO:teuthology.orchestra.run.trial195.stdout:(148/158): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2026-02-26T03:49:12.491 INFO:teuthology.orchestra.run.trial195.stdout:(149/158): python3-xmlsec-1.3.13-1.el9.x86_64.r 1.4 MB/s | 48 kB 00:00 2026-02-26T03:49:12.525 INFO:teuthology.orchestra.run.trial195.stdout:(150/158): python3-xmltodict-0.12.0-15.el9.noar 654 kB/s | 22 kB 00:00 2026-02-26T03:49:12.559 INFO:teuthology.orchestra.run.trial195.stdout:(151/158): python3-zc-lockfile-2.0-10.el9.noarc 594 kB/s | 20 kB 00:00 2026-02-26T03:49:12.593 INFO:teuthology.orchestra.run.trial195.stdout:(152/158): re2-20211101-20.el9.x86_64.rpm 5.5 MB/s | 191 kB 00:00 2026-02-26T03:49:12.658 INFO:teuthology.orchestra.run.trial195.stdout:(153/158): thrift-0.15.0-4.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2026-02-26T03:49:12.692 INFO:teuthology.orchestra.run.trial195.stdout:(154/158): yaml-cpp-0.6.3-5.el9.x86_64.rpm 3.4 MB/s | 122 kB 00:00 2026-02-26T03:49:13.104 INFO:teuthology.orchestra.run.trial195.stdout:(155/158): librados2-20.3.0-5312.gabffb617.el9. 8.0 MB/s | 3.3 MB 00:00 2026-02-26T03:49:13.296 INFO:teuthology.orchestra.run.trial195.stdout:(156/158): librbd1-20.3.0-5312.gabffb617.el9.x8 14 MB/s | 2.7 MB 00:00 2026-02-26T03:49:14.290 INFO:teuthology.orchestra.run.trial195.stdout:(157/158): ceph-osd-crimson-debuginfo-20.3.0-53 24 MB/s | 429 MB 00:17 2026-02-26T03:49:14.361 INFO:teuthology.orchestra.run.trial195.stdout:(158/158): ceph-mgr-dashboard-20.3.0-5312.gabff 6.8 MB/s | 51 MB 00:07 2026-02-26T03:49:14.363 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-26T03:49:14.364 INFO:teuthology.orchestra.run.trial195.stdout:Total 45 MB/s | 863 MB 00:19 2026-02-26T03:49:14.954 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-26T03:49:15.007 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-26T03:49:15.007 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-26T03:49:17.747 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-26T03:49:17.747 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-26T03:49:20.554 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-26T03:49:20.557 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/160 2026-02-26T03:49:20.565 INFO:teuthology.orchestra.run.trial195.stdout: Installing : liboath-2.6.12-1.el9.x86_64 2/160 2026-02-26T03:49:20.577 INFO:teuthology.orchestra.run.trial195.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/160 2026-02-26T03:49:20.719 INFO:teuthology.orchestra.run.trial195.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2026-02-26T03:49:20.721 INFO:teuthology.orchestra.run.trial195.stdout: Upgrading : librados2-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:49:20.767 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librados2-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:49:20.768 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 6/160 2026-02-26T03:49:20.781 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 6/160 2026-02-26T03:49:20.783 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86 7/160 2026-02-26T03:49:20.784 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 8/160 2026-02-26T03:49:20.811 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 8/160 2026-02-26T03:49:20.822 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 9/160 2026-02-26T03:49:20.836 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 10/160 2026-02-26T03:49:20.838 INFO:teuthology.orchestra.run.trial195.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 11/160 2026-02-26T03:49:20.841 INFO:teuthology.orchestra.run.trial195.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 12/160 2026-02-26T03:49:20.843 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 13/160 2026-02-26T03:49:20.847 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 14/160 2026-02-26T03:49:20.963 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 15/160 2026-02-26T03:49:20.964 INFO:teuthology.orchestra.run.trial195.stdout: Upgrading : librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:49:21.010 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:49:21.016 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 17/160 2026-02-26T03:49:21.025 INFO:teuthology.orchestra.run.trial195.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 18/160 2026-02-26T03:49:21.025 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 19/160 2026-02-26T03:49:21.049 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 19/160 2026-02-26T03:49:21.050 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 20/160 2026-02-26T03:49:21.062 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 20/160 2026-02-26T03:49:21.066 INFO:teuthology.orchestra.run.trial195.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 21/160 2026-02-26T03:49:21.097 INFO:teuthology.orchestra.run.trial195.stdout: Installing : re2-1:20211101-20.el9.x86_64 22/160 2026-02-26T03:49:21.103 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 23/160 2026-02-26T03:49:21.106 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 24/160 2026-02-26T03:49:21.128 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 25/160 2026-02-26T03:49:21.133 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 26/160 2026-02-26T03:49:21.136 INFO:teuthology.orchestra.run.trial195.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2026-02-26T03:49:21.140 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 28/160 2026-02-26T03:49:21.141 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 29/160 2026-02-26T03:49:21.153 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 29/160 2026-02-26T03:49:21.158 INFO:teuthology.orchestra.run.trial195.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 30/160 2026-02-26T03:49:21.181 INFO:teuthology.orchestra.run.trial195.stdout: Installing : unzip-6.0-59.el9.x86_64 31/160 2026-02-26T03:49:21.194 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 32/160 2026-02-26T03:49:21.200 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 33/160 2026-02-26T03:49:21.203 INFO:teuthology.orchestra.run.trial195.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 34/160 2026-02-26T03:49:21.209 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 35/160 2026-02-26T03:49:21.211 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 36/160 2026-02-26T03:49:21.212 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 37/160 2026-02-26T03:49:21.222 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 38/160 2026-02-26T03:49:21.256 INFO:teuthology.orchestra.run.trial195.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 39/160 2026-02-26T03:49:21.261 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-ceph-common-2:20.3.0-5312.gabffb617.el9. 40/160 2026-02-26T03:49:21.271 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-ceph-argparse-2:20.3.0-5312.gabffb617.el 41/160 2026-02-26T03:49:21.283 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_6 42/160 2026-02-26T03:49:21.289 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 43/160 2026-02-26T03:49:21.313 INFO:teuthology.orchestra.run.trial195.stdout: Installing : zip-3.0-35.el9.x86_64 44/160 2026-02-26T03:49:21.316 INFO:teuthology.orchestra.run.trial195.stdout: Installing : luarocks-3.9.2-5.el9.noarch 45/160 2026-02-26T03:49:21.366 INFO:teuthology.orchestra.run.trial195.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 46/160 2026-02-26T03:49:21.379 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 47/160 2026-02-26T03:49:21.385 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-rsa-4.9-2.el9.noarch 48/160 2026-02-26T03:49:21.389 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/160 2026-02-26T03:49:21.390 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/160 2026-02-26T03:49:21.395 INFO:teuthology.orchestra.run.trial195.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 51/160 2026-02-26T03:49:21.411 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 52/160 2026-02-26T03:49:21.416 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 53/160 2026-02-26T03:49:21.420 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/160 2026-02-26T03:49:21.427 INFO:teuthology.orchestra.run.trial195.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 55/160 2026-02-26T03:49:21.435 INFO:teuthology.orchestra.run.trial195.stdout: Installing : librados-devel-2:20.3.0-5312.gabffb617.el9.x86_6 56/160 2026-02-26T03:49:21.447 INFO:teuthology.orchestra.run.trial195.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 57/160 2026-02-26T03:49:21.451 INFO:teuthology.orchestra.run.trial195.stdout: Installing : thrift-0.15.0-4.el9.x86_64 58/160 2026-02-26T03:49:21.455 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 59/160 2026-02-26T03:49:21.469 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 60/160 2026-02-26T03:49:21.473 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 61/160 2026-02-26T03:49:21.478 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 62/160 2026-02-26T03:49:21.490 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 63/160 2026-02-26T03:49:21.498 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 64/160 2026-02-26T03:49:21.505 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 65/160 2026-02-26T03:49:21.524 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 66/160 2026-02-26T03:49:21.536 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 67/160 2026-02-26T03:49:21.551 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 68/160 2026-02-26T03:49:21.556 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 69/160 2026-02-26T03:49:21.563 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 70/160 2026-02-26T03:49:21.603 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 71/160 2026-02-26T03:49:21.927 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 72/160 2026-02-26T03:49:21.936 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 73/160 2026-02-26T03:49:21.940 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 74/160 2026-02-26T03:49:21.946 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 75/160 2026-02-26T03:49:21.952 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 76/160 2026-02-26T03:49:21.955 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 77/160 2026-02-26T03:49:21.960 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/160 2026-02-26T03:49:21.966 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 79/160 2026-02-26T03:49:21.969 INFO:teuthology.orchestra.run.trial195.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 80/160 2026-02-26T03:49:21.970 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 81/160 2026-02-26T03:49:21.999 INFO:teuthology.orchestra.run.trial195.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 82/160 2026-02-26T03:49:22.044 INFO:teuthology.orchestra.run.trial195.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 83/160 2026-02-26T03:49:22.053 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 84/160 2026-02-26T03:49:22.100 INFO:teuthology.orchestra.run.trial195.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 85/160 2026-02-26T03:49:22.128 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-py-1.10.0-6.el9.noarch 86/160 2026-02-26T03:49:22.138 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 87/160 2026-02-26T03:49:22.145 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 88/160 2026-02-26T03:49:22.150 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 89/160 2026-02-26T03:49:22.157 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 90/160 2026-02-26T03:49:22.161 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 91/160 2026-02-26T03:49:22.196 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 92/160 2026-02-26T03:49:22.418 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 93/160 2026-02-26T03:49:22.442 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 94/160 2026-02-26T03:49:22.446 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 95/160 2026-02-26T03:49:22.504 INFO:teuthology.orchestra.run.trial195.stdout: Installing : openblas-0.3.29-1.el9.x86_64 96/160 2026-02-26T03:49:22.506 INFO:teuthology.orchestra.run.trial195.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 97/160 2026-02-26T03:49:22.528 INFO:teuthology.orchestra.run.trial195.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 98/160 2026-02-26T03:49:22.853 INFO:teuthology.orchestra.run.trial195.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 99/160 2026-02-26T03:49:22.926 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 100/160 2026-02-26T03:49:23.596 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 101/160 2026-02-26T03:49:23.610 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 102/160 2026-02-26T03:49:23.616 INFO:teuthology.orchestra.run.trial195.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 103/160 2026-02-26T03:49:23.863 INFO:teuthology.orchestra.run.trial195.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 104/160 2026-02-26T03:49:23.865 INFO:teuthology.orchestra.run.trial195.stdout: Installing : librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 105/160 2026-02-26T03:49:23.883 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 105/160 2026-02-26T03:49:23.885 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 106/160 2026-02-26T03:49:24.826 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:49:24.829 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:49:24.846 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:49:24.856 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 108/160 2026-02-26T03:49:24.863 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-packaging-20.9-5.el9.noarch 109/160 2026-02-26T03:49:24.876 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-ply-3.11-14.el9.noarch 110/160 2026-02-26T03:49:24.892 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 111/160 2026-02-26T03:49:24.968 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 112/160 2026-02-26T03:49:24.976 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 113/160 2026-02-26T03:49:25.000 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 114/160 2026-02-26T03:49:25.029 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 115/160 2026-02-26T03:49:25.080 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 116/160 2026-02-26T03:49:25.086 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 117/160 2026-02-26T03:49:25.089 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 118/160 2026-02-26T03:49:25.094 INFO:teuthology.orchestra.run.trial195.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 119/160 2026-02-26T03:49:25.097 INFO:teuthology.orchestra.run.trial195.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 120/160 2026-02-26T03:49:25.097 INFO:teuthology.orchestra.run.trial195.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 121/160 2026-02-26T03:49:25.114 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 121/160 2026-02-26T03:49:25.229 INFO:teuthology.orchestra.run.trial195.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 122/160 2026-02-26T03:49:25.231 INFO:teuthology.orchestra.run.trial195.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 123/160 2026-02-26T03:49:25.513 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 123/160 2026-02-26T03:49:25.513 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-26T03:49:25.513 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:49:25.772 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 124/160 2026-02-26T03:49:25.814 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 124/160 2026-02-26T03:49:25.814 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-26T03:49:25.814 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-26T03:49:25.814 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:49:25.818 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 125/160 2026-02-26T03:49:27.458 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - CRB 430 kB/s | 8.0 MB 00:18 2026-02-26T03:49:28.323 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - Extras packages 169 kB/s | 20 kB 00:00 2026-02-26T03:49:29.002 INFO:teuthology.orchestra.run.trial160.stdout:Extra Packages for Enterprise Linux 33 MB/s | 20 MB 00:00 2026-02-26T03:49:30.469 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 125/160 2026-02-26T03:49:30.469 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /sys 2026-02-26T03:49:30.469 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /proc 2026-02-26T03:49:30.469 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /mnt 2026-02-26T03:49:30.470 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /var/tmp 2026-02-26T03:49:30.470 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /home 2026-02-26T03:49:30.470 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /root 2026-02-26T03:49:30.470 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /tmp 2026-02-26T03:49:30.470 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:49:30.557 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 126/160 2026-02-26T03:49:30.580 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 126/160 2026-02-26T03:49:30.580 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:49:30.580 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-26T03:49:30.580 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-26T03:49:30.580 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-26T03:49:30.580 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:49:30.786 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 127/160 2026-02-26T03:49:30.807 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 127/160 2026-02-26T03:49:30.807 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:49:30.807 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-26T03:49:30.807 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-26T03:49:30.807 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-26T03:49:30.807 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:49:30.812 INFO:teuthology.orchestra.run.trial195.stdout: Installing : mailcap-2.1.49-5.el9.noarch 128/160 2026-02-26T03:49:30.814 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 129/160 2026-02-26T03:49:30.833 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:49:30.834 INFO:teuthology.orchestra.run.trial195.stdout:Creating group 'qat' with GID 992. 2026-02-26T03:49:30.834 INFO:teuthology.orchestra.run.trial195.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-26T03:49:30.834 INFO:teuthology.orchestra.run.trial195.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-26T03:49:30.834 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:49:30.842 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:49:30.866 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:49:30.866 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-26T03:49:30.866 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:49:30.901 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 131/160 2026-02-26T03:49:31.489 INFO:teuthology.orchestra.run.trial195.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 132/160 2026-02-26T03:49:32.209 INFO:teuthology.orchestra.run.trial160.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-26T03:49:33.437 INFO:teuthology.orchestra.run.trial160.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-26T03:49:33.438 INFO:teuthology.orchestra.run.trial160.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-26T03:49:33.466 INFO:teuthology.orchestra.run.trial160.stdout:Dependencies resolved. 2026-02-26T03:49:33.471 INFO:teuthology.orchestra.run.trial160.stdout:======================================================================================= 2026-02-26T03:49:33.471 INFO:teuthology.orchestra.run.trial160.stdout: Package Arch Version Repository Size 2026-02-26T03:49:33.471 INFO:teuthology.orchestra.run.trial160.stdout:======================================================================================= 2026-02-26T03:49:33.471 INFO:teuthology.orchestra.run.trial160.stdout:Installing: 2026-02-26T03:49:33.471 INFO:teuthology.orchestra.run.trial160.stdout: ceph x86_64 2:20.3.0-5312.gabffb617.el9 ceph 6.5 k 2026-02-26T03:49:33.471 INFO:teuthology.orchestra.run.trial160.stdout: ceph-base x86_64 2:20.3.0-5312.gabffb617.el9 ceph 5.1 M 2026-02-26T03:49:33.471 INFO:teuthology.orchestra.run.trial160.stdout: ceph-fuse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 914 k 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5312.gabffb617.el9 ceph 149 k 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr x86_64 2:20.3.0-5312.gabffb617.el9 ceph 937 k 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 184 k 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 51 M 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 7.4 M 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-rook noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 50 k 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-osd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 163 k 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-osd-classic x86_64 2:20.3.0-5312.gabffb617.el9 ceph 15 M 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-osd-crimson x86_64 2:20.3.0-5312.gabffb617.el9 ceph 87 M 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5312.gabffb617.el9 ceph 429 M 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-radosgw x86_64 2:20.3.0-5312.gabffb617.el9 ceph 11 M 2026-02-26T03:49:33.472 INFO:teuthology.orchestra.run.trial160.stdout: ceph-test x86_64 2:20.3.0-5312.gabffb617.el9 ceph 126 M 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: ceph-volume noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 301 k 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: cephadm noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 1.0 M 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: libcephfs-devel x86_64 2:20.3.0-5312.gabffb617.el9 ceph 37 k 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: libcephfs2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 859 k 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: librados-devel x86_64 2:20.3.0-5312.gabffb617.el9 ceph 124 k 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: python3-cephfs x86_64 2:20.3.0-5312.gabffb617.el9 ceph 203 k 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: python3-pytest noarch 6.2.2-7.el9 appstream 519 k 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: python3-rados x86_64 2:20.3.0-5312.gabffb617.el9 ceph 324 k 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: python3-rbd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 305 k 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: python3-rgw x86_64 2:20.3.0-5312.gabffb617.el9 ceph 100 k 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: rbd-fuse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 87 k 2026-02-26T03:49:33.473 INFO:teuthology.orchestra.run.trial160.stdout: rbd-mirror x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.6 M 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout: rbd-nbd x86_64 2:20.3.0-5312.gabffb617.el9 ceph 166 k 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout:Upgrading: 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout: librados2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 3.3 M 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout: librbd1 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.7 M 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout:Installing dependencies: 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout: ceph-common x86_64 2:20.3.0-5312.gabffb617.el9 ceph 21 M 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 57 k 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mds x86_64 2:20.3.0-5312.gabffb617.el9 ceph 2.0 M 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 297 k 2026-02-26T03:49:33.474 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mon x86_64 2:20.3.0-5312.gabffb617.el9 ceph 4.9 M 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 17 k 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: ceph-selinux x86_64 2:20.3.0-5312.gabffb617.el9 ceph 25 k 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-26T03:49:33.475 INFO:teuthology.orchestra.run.trial160.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 26 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: libcephsqlite x86_64 2:20.3.0-5312.gabffb617.el9 ceph 153 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: libradosstriper1 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 263 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-26T03:49:33.476 INFO:teuthology.orchestra.run.trial160.stdout: librgw2 x86_64 2:20.3.0-5312.gabffb617.el9 ceph 5.6 M 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-26T03:49:33.477 INFO:teuthology.orchestra.run.trial160.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: python3-ceph-argparse x86_64 2:20.3.0-5312.gabffb617.el9 ceph 45 k 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: python3-ceph-common x86_64 2:20.3.0-5312.gabffb617.el9 ceph 185 k 2026-02-26T03:49:33.478 INFO:teuthology.orchestra.run.trial160.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-iniconfig noarch 1.1.1-7.el9 appstream 17 k 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-26T03:49:33.479 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-26T03:49:33.480 INFO:teuthology.orchestra.run.trial160.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-pluggy noarch 0.13.1-7.el9 appstream 41 k 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-py noarch 1.10.0-6.el9 appstream 477 k 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-26T03:49:33.481 INFO:teuthology.orchestra.run.trial160.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-26T03:49:33.482 INFO:teuthology.orchestra.run.trial160.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-26T03:49:33.483 INFO:teuthology.orchestra.run.trial160.stdout:Installing weak dependencies: 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout: ceph-debugsource x86_64 2:20.3.0-5312.gabffb617.el9 ceph 12 M 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5312.gabffb617.el9 ceph-noarch 22 k 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout: libcephfs-daemon x86_64 2:20.3.0-5312.gabffb617.el9 ceph 37 k 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout:Transaction Summary 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout:======================================================================================= 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout:Install 156 Packages 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout:Upgrade 2 Packages 2026-02-26T03:49:33.484 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:49:33.485 INFO:teuthology.orchestra.run.trial160.stdout:Total download size: 863 M 2026-02-26T03:49:33.485 INFO:teuthology.orchestra.run.trial160.stdout:Downloading Packages: 2026-02-26T03:49:34.442 INFO:teuthology.orchestra.run.trial160.stdout:(1/158): ceph-20.3.0-5312.gabffb617.el9.x86_64. 27 kB/s | 6.5 kB 00:00 2026-02-26T03:49:35.201 INFO:teuthology.orchestra.run.trial160.stdout:(2/158): ceph-base-20.3.0-5312.gabffb617.el9.x8 5.1 MB/s | 5.1 MB 00:00 2026-02-26T03:49:35.302 INFO:teuthology.orchestra.run.trial160.stdout:(3/158): ceph-fuse-20.3.0-5312.gabffb617.el9.x8 8.9 MB/s | 914 kB 00:00 2026-02-26T03:49:35.348 INFO:teuthology.orchestra.run.trial160.stdout:(4/158): ceph-immutable-object-cache-20.3.0-531 3.2 MB/s | 149 kB 00:00 2026-02-26T03:49:35.566 INFO:teuthology.orchestra.run.trial160.stdout:(5/158): ceph-mds-20.3.0-5312.gabffb617.el9.x86 9.3 MB/s | 2.0 MB 00:00 2026-02-26T03:49:35.680 INFO:teuthology.orchestra.run.trial160.stdout:(6/158): ceph-mgr-20.3.0-5312.gabffb617.el9.x86 8.0 MB/s | 937 kB 00:00 2026-02-26T03:49:35.788 INFO:teuthology.orchestra.run.trial160.stdout:(7/158): ceph-debugsource-20.3.0-5312.gabffb617 8.9 MB/s | 12 MB 00:01 2026-02-26T03:49:35.841 INFO:teuthology.orchestra.run.trial160.stdout:(8/158): ceph-osd-20.3.0-5312.gabffb617.el9.x86 3.0 MB/s | 163 kB 00:00 2026-02-26T03:49:36.139 INFO:teuthology.orchestra.run.trial160.stdout:(9/158): ceph-mon-20.3.0-5312.gabffb617.el9.x86 11 MB/s | 4.9 MB 00:00 2026-02-26T03:49:36.813 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 133/160 2026-02-26T03:49:36.876 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 134/160 2026-02-26T03:49:36.878 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 135/160 2026-02-26T03:49:36.891 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 135/160 2026-02-26T03:49:36.891 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:49:36.891 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-26T03:49:36.891 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:49:36.901 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 136/160 2026-02-26T03:49:36.924 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 136/160 2026-02-26T03:49:36.924 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:49:36.924 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-26T03:49:36.924 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-26T03:49:36.925 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-26T03:49:36.925 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:49:36.984 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: cephadm-2:20.3.0-5312.gabffb617.el9.noarch 137/160 2026-02-26T03:49:36.985 INFO:teuthology.orchestra.run.trial195.stdout: Installing : cephadm-2:20.3.0-5312.gabffb617.el9.noarch 137/160 2026-02-26T03:49:36.995 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.e 138/160 2026-02-26T03:49:37.014 INFO:teuthology.orchestra.run.trial160.stdout:(10/158): ceph-osd-classic-20.3.0-5312.gabffb61 13 MB/s | 15 MB 00:01 2026-02-26T03:49:37.018 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5312.gabffb617. 139/160 2026-02-26T03:49:37.019 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 140/160 2026-02-26T03:49:39.890 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 140/160 2026-02-26T03:49:39.897 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 141/160 2026-02-26T03:49:40.350 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 141/160 2026-02-26T03:49:40.351 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 142/160 2026-02-26T03:49:40.363 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 142/160 2026-02-26T03:49:40.363 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 143/160 2026-02-26T03:49:40.424 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 143/160 2026-02-26T03:49:40.464 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el 144/160 2026-02-26T03:49:40.465 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 145/160 2026-02-26T03:49:40.486 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 145/160 2026-02-26T03:49:40.486 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:49:40.487 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-26T03:49:40.487 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-26T03:49:40.487 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-26T03:49:40.487 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:49:40.498 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 146/160 2026-02-26T03:49:41.091 INFO:teuthology.orchestra.run.trial160.stdout:(11/158): ceph-common-20.3.0-5312.gabffb617.el9 3.0 MB/s | 21 MB 00:06 2026-02-26T03:49:42.753 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 146/160 2026-02-26T03:49:44.320 INFO:teuthology.orchestra.run.trial160.stdout:(12/158): ceph-radosgw-20.3.0-5312.gabffb617.el 3.5 MB/s | 11 MB 00:03 2026-02-26T03:49:44.384 INFO:teuthology.orchestra.run.trial160.stdout:(13/158): ceph-selinux-20.3.0-5312.gabffb617.el 392 kB/s | 25 kB 00:00 2026-02-26T03:49:44.531 INFO:teuthology.orchestra.run.trial160.stdout:(14/158): ceph-osd-crimson-20.3.0-5312.gabffb61 10 MB/s | 87 MB 00:08 2026-02-26T03:49:44.589 INFO:teuthology.orchestra.run.trial160.stdout:(15/158): libcephfs-daemon-20.3.0-5312.gabffb61 641 kB/s | 37 kB 00:00 2026-02-26T03:49:44.647 INFO:teuthology.orchestra.run.trial160.stdout:(16/158): libcephfs-devel-20.3.0-5312.gabffb617 645 kB/s | 37 kB 00:00 2026-02-26T03:49:44.705 INFO:teuthology.orchestra.run.trial160.stdout:(17/158): libcephfs-proxy2-20.3.0-5312.gabffb61 444 kB/s | 26 kB 00:00 2026-02-26T03:49:44.834 INFO:teuthology.orchestra.run.trial160.stdout:(18/158): libcephfs2-20.3.0-5312.gabffb617.el9. 6.5 MB/s | 859 kB 00:00 2026-02-26T03:49:44.895 INFO:teuthology.orchestra.run.trial160.stdout:(19/158): libcephsqlite-20.3.0-5312.gabffb617.e 2.5 MB/s | 153 kB 00:00 2026-02-26T03:49:44.955 INFO:teuthology.orchestra.run.trial160.stdout:(20/158): librados-devel-20.3.0-5312.gabffb617. 2.0 MB/s | 124 kB 00:00 2026-02-26T03:49:45.018 INFO:teuthology.orchestra.run.trial160.stdout:(21/158): libradosstriper1-20.3.0-5312.gabffb61 4.1 MB/s | 263 kB 00:00 2026-02-26T03:49:45.753 INFO:teuthology.orchestra.run.trial160.stdout:(22/158): librgw2-20.3.0-5312.gabffb617.el9.x86 7.7 MB/s | 5.6 MB 00:00 2026-02-26T03:49:45.807 INFO:teuthology.orchestra.run.trial160.stdout:(23/158): python3-ceph-argparse-20.3.0-5312.gab 841 kB/s | 45 kB 00:00 2026-02-26T03:49:45.867 INFO:teuthology.orchestra.run.trial160.stdout:(24/158): python3-ceph-common-20.3.0-5312.gabff 3.0 MB/s | 185 kB 00:00 2026-02-26T03:49:45.928 INFO:teuthology.orchestra.run.trial160.stdout:(25/158): python3-cephfs-20.3.0-5312.gabffb617. 3.3 MB/s | 203 kB 00:00 2026-02-26T03:49:45.992 INFO:teuthology.orchestra.run.trial160.stdout:(26/158): python3-rados-20.3.0-5312.gabffb617.e 4.9 MB/s | 324 kB 00:00 2026-02-26T03:49:46.056 INFO:teuthology.orchestra.run.trial160.stdout:(27/158): python3-rbd-20.3.0-5312.gabffb617.el9 4.7 MB/s | 305 kB 00:00 2026-02-26T03:49:46.116 INFO:teuthology.orchestra.run.trial160.stdout:(28/158): python3-rgw-20.3.0-5312.gabffb617.el9 1.6 MB/s | 100 kB 00:00 2026-02-26T03:49:46.175 INFO:teuthology.orchestra.run.trial160.stdout:(29/158): rbd-fuse-20.3.0-5312.gabffb617.el9.x8 1.4 MB/s | 87 kB 00:00 2026-02-26T03:49:46.497 INFO:teuthology.orchestra.run.trial160.stdout:(30/158): rbd-mirror-20.3.0-5312.gabffb617.el9. 8.1 MB/s | 2.6 MB 00:00 2026-02-26T03:49:46.553 INFO:teuthology.orchestra.run.trial160.stdout:(31/158): rbd-nbd-20.3.0-5312.gabffb617.el9.x86 2.9 MB/s | 166 kB 00:00 2026-02-26T03:49:46.607 INFO:teuthology.orchestra.run.trial160.stdout:(32/158): ceph-grafana-dashboards-20.3.0-5312.g 1.0 MB/s | 57 kB 00:00 2026-02-26T03:49:46.663 INFO:teuthology.orchestra.run.trial160.stdout:(33/158): ceph-mgr-cephadm-20.3.0-5312.gabffb61 3.2 MB/s | 184 kB 00:00 2026-02-26T03:49:52.636 INFO:teuthology.orchestra.run.trial160.stdout:(34/158): ceph-mgr-dashboard-20.3.0-5312.gabffb 8.5 MB/s | 51 MB 00:05 2026-02-26T03:49:53.650 INFO:teuthology.orchestra.run.trial160.stdout:(35/158): ceph-mgr-diskprediction-local-20.3.0- 7.3 MB/s | 7.4 MB 00:01 2026-02-26T03:49:53.707 INFO:teuthology.orchestra.run.trial160.stdout:(36/158): ceph-mgr-k8sevents-20.3.0-5312.gabffb 391 kB/s | 22 kB 00:00 2026-02-26T03:49:53.770 INFO:teuthology.orchestra.run.trial160.stdout:(37/158): ceph-mgr-modules-core-20.3.0-5312.gab 4.6 MB/s | 297 kB 00:00 2026-02-26T03:49:53.827 INFO:teuthology.orchestra.run.trial160.stdout:(38/158): ceph-mgr-rook-20.3.0-5312.gabffb617.e 874 kB/s | 50 kB 00:00 2026-02-26T03:49:53.884 INFO:teuthology.orchestra.run.trial160.stdout:(39/158): ceph-prometheus-alerts-20.3.0-5312.ga 309 kB/s | 17 kB 00:00 2026-02-26T03:49:53.944 INFO:teuthology.orchestra.run.trial160.stdout:(40/158): ceph-volume-20.3.0-5312.gabffb617.el9 4.9 MB/s | 301 kB 00:00 2026-02-26T03:49:54.129 INFO:teuthology.orchestra.run.trial160.stdout:(41/158): cephadm-20.3.0-5312.gabffb617.el9.noa 5.5 MB/s | 1.0 MB 00:00 2026-02-26T03:49:54.985 INFO:teuthology.orchestra.run.trial160.stdout:(42/158): cryptsetup-2.8.1-3.el9.x86_64.rpm 410 kB/s | 351 kB 00:00 2026-02-26T03:49:56.042 INFO:teuthology.orchestra.run.trial160.stdout:(43/158): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.0 MB/s | 2.1 MB 00:01 2026-02-26T03:49:56.646 INFO:teuthology.orchestra.run.trial160.stdout:(44/158): ledmon-libs-1.1.0-3.el9.x86_64.rpm 67 kB/s | 40 kB 00:00 2026-02-26T03:49:57.210 INFO:teuthology.orchestra.run.trial160.stdout:(45/158): libatomic-11.5.0-14.el9.x86_64.rpm 40 kB/s | 23 kB 00:00 2026-02-26T03:49:57.874 INFO:teuthology.orchestra.run.trial160.stdout:(46/158): libconfig-1.7.2-9.el9.x86_64.rpm 108 kB/s | 72 kB 00:00 2026-02-26T03:49:58.823 INFO:teuthology.orchestra.run.trial160.stdout:(47/158): libgfortran-11.5.0-14.el9.x86_64.rpm 837 kB/s | 794 kB 00:00 2026-02-26T03:49:59.584 INFO:teuthology.orchestra.run.trial160.stdout:(48/158): libquadmath-11.5.0-14.el9.x86_64.rpm 242 kB/s | 184 kB 00:00 2026-02-26T03:50:00.289 INFO:teuthology.orchestra.run.trial160.stdout:(49/158): lksctp-tools-1.0.19-2.el9.x86_64.rpm 134 kB/s | 94 kB 00:00 2026-02-26T03:50:00.880 INFO:teuthology.orchestra.run.trial160.stdout:(50/158): mailcap-2.1.49-5.el9.noarch.rpm 56 kB/s | 33 kB 00:00 2026-02-26T03:50:01.879 INFO:teuthology.orchestra.run.trial160.stdout:(51/158): nvme-cli-2.16-1.el9.x86_64.rpm 1.2 MB/s | 1.2 MB 00:00 2026-02-26T03:50:02.572 INFO:teuthology.orchestra.run.trial160.stdout:(52/158): pciutils-3.7.0-7.el9.x86_64.rpm 134 kB/s | 93 kB 00:00 2026-02-26T03:50:03.382 INFO:teuthology.orchestra.run.trial160.stdout:(53/158): python3-cffi-1.14.5-5.el9.x86_64.rpm 313 kB/s | 253 kB 00:00 2026-02-26T03:50:04.357 INFO:teuthology.orchestra.run.trial160.stdout:(54/158): python3-cryptography-36.0.1-5.el9.x86 1.3 MB/s | 1.2 MB 00:00 2026-02-26T03:50:04.790 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86 147/160 2026-02-26T03:50:04.791 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb6 148/160 2026-02-26T03:50:05.052 INFO:teuthology.orchestra.run.trial160.stdout:(55/158): python3-ply-3.11-14.el9.noarch.rpm 153 kB/s | 106 kB 00:00 2026-02-26T03:50:05.272 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-2:20.3.0-5312.gabffb617.el9.x86_64 149/160 2026-02-26T03:50:05.273 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 150/160 2026-02-26T03:50:05.313 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 150/160 2026-02-26T03:50:05.314 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:50:05.314 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-26T03:50:05.314 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-26T03:50:05.314 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-26T03:50:05.314 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:50:05.323 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5312.gabffb 151/160 2026-02-26T03:50:05.343 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5312.gabffb 151/160 2026-02-26T03:50:05.343 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:50:05.344 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-26T03:50:05.344 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:50:05.453 INFO:teuthology.orchestra.run.trial195.stdout: Installing : rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 152/160 2026-02-26T03:50:05.475 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 152/160 2026-02-26T03:50:05.475 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:50:05.475 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-26T03:50:05.475 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-26T03:50:05.475 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-26T03:50:05.476 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:50:05.906 INFO:teuthology.orchestra.run.trial160.stdout:(56/158): ceph-test-20.3.0-5312.gabffb617.el9.x 5.9 MB/s | 126 MB 00:21 2026-02-26T03:50:05.907 INFO:teuthology.orchestra.run.trial160.stdout:(57/158): python3-pycparser-2.20-6.el9.noarch.r 158 kB/s | 135 kB 00:00 2026-02-26T03:50:07.246 INFO:teuthology.orchestra.run.trial160.stdout:(58/158): ceph-osd-crimson-debuginfo-20.3.0-531 14 MB/s | 429 MB 00:30 2026-02-26T03:50:07.247 INFO:teuthology.orchestra.run.trial160.stdout:(59/158): python3-requests-2.25.1-10.el9.noarch 94 kB/s | 126 kB 00:01 2026-02-26T03:50:07.248 INFO:teuthology.orchestra.run.trial160.stdout:(60/158): python3-pyparsing-2.4.7-9.el9.noarch. 112 kB/s | 150 kB 00:01 2026-02-26T03:50:08.021 INFO:teuthology.orchestra.run.trial160.stdout:(61/158): unzip-6.0-59.el9.x86_64.rpm 235 kB/s | 182 kB 00:00 2026-02-26T03:50:08.036 INFO:teuthology.orchestra.run.trial160.stdout:(62/158): python3-urllib3-1.26.5-7.el9.noarch.r 276 kB/s | 218 kB 00:00 2026-02-26T03:50:08.045 INFO:teuthology.orchestra.run.trial160.stdout:(63/158): zip-3.0-35.el9.x86_64.rpm 334 kB/s | 266 kB 00:00 2026-02-26T03:50:08.617 INFO:teuthology.orchestra.run.trial160.stdout:(64/158): flexiblas-3.0.4-9.el9.x86_64.rpm 51 kB/s | 30 kB 00:00 2026-02-26T03:50:08.730 INFO:teuthology.orchestra.run.trial160.stdout:(65/158): boost-program-options-1.75.0-13.el9.x 147 kB/s | 104 kB 00:00 2026-02-26T03:50:09.113 INFO:teuthology.orchestra.run.trial160.stdout:(66/158): flexiblas-openblas-openmp-3.0.4-9.el9 30 kB/s | 15 kB 00:00 2026-02-26T03:50:09.124 INFO:teuthology.orchestra.run.trial160.stdout:(67/158): flexiblas-netlib-3.0.4-9.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2026-02-26T03:50:09.629 INFO:teuthology.orchestra.run.trial160.stdout:(68/158): libasan8-15.2.1-7.el9.x86_64.rpm 584 kB/s | 525 kB 00:00 2026-02-26T03:50:09.874 INFO:teuthology.orchestra.run.trial160.stdout:(69/158): libnbd-1.20.3-4.el9.x86_64.rpm 215 kB/s | 164 kB 00:00 2026-02-26T03:50:09.879 INFO:teuthology.orchestra.run.trial160.stdout:(70/158): libpmemobj-1.12.1-1.el9.x86_64.rpm 212 kB/s | 160 kB 00:00 2026-02-26T03:50:10.231 INFO:teuthology.orchestra.run.trial160.stdout:(71/158): librabbitmq-0.11.0-7.el9.x86_64.rpm 75 kB/s | 45 kB 00:00 2026-02-26T03:50:10.667 INFO:teuthology.orchestra.run.trial160.stdout:(72/158): libstoragemgmt-1.10.1-1.el9.x86_64.rp 313 kB/s | 246 kB 00:00 2026-02-26T03:50:10.796 INFO:teuthology.orchestra.run.trial160.stdout:(73/158): librdkafka-1.6.1-102.el9.x86_64.rpm 719 kB/s | 662 kB 00:00 2026-02-26T03:50:10.986 INFO:teuthology.orchestra.run.trial160.stdout:(74/158): libubsan-11.5.0-14.el9.x86_64.rpm 233 kB/s | 176 kB 00:00 2026-02-26T03:50:11.468 INFO:teuthology.orchestra.run.trial160.stdout:(75/158): libxslt-1.1.34-12.el9.x86_64.rpm 290 kB/s | 233 kB 00:00 2026-02-26T03:50:11.625 INFO:teuthology.orchestra.run.trial160.stdout:(76/158): lttng-ust-2.12.0-6.el9.x86_64.rpm 353 kB/s | 292 kB 00:00 2026-02-26T03:50:11.768 INFO:teuthology.orchestra.run.trial160.stdout:(77/158): lua-5.4.4-4.el9.x86_64.rpm 241 kB/s | 188 kB 00:00 2026-02-26T03:50:12.078 INFO:teuthology.orchestra.run.trial160.stdout:(78/158): openblas-0.3.29-1.el9.x86_64.rpm 69 kB/s | 42 kB 00:00 2026-02-26T03:50:12.389 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pytest-6.2.2-7.el9.noarch 153/160 2026-02-26T03:50:12.430 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 154/160 2026-02-26T03:50:12.450 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_ 155/160 2026-02-26T03:50:12.469 INFO:teuthology.orchestra.run.trial195.stdout: Installing : rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 156/160 2026-02-26T03:50:12.510 INFO:teuthology.orchestra.run.trial195.stdout: Installing : rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 157/160 2026-02-26T03:50:12.513 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 158/160 2026-02-26T03:50:12.513 INFO:teuthology.orchestra.run.trial195.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/160 2026-02-26T03:50:12.526 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/160 2026-02-26T03:50:12.526 INFO:teuthology.orchestra.run.trial195.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:50:12.538 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:50:12.544 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 160/160 2026-02-26T03:50:12.570 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 160/160 2026-02-26T03:50:12.723 INFO:teuthology.orchestra.run.trial160.stdout:(79/158): protobuf-3.14.0-17.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2026-02-26T03:50:12.797 INFO:teuthology.orchestra.run.trial160.stdout:(80/158): openblas-openmp-0.3.29-1.el9.x86_64.r 4.5 MB/s | 5.3 MB 00:01 2026-02-26T03:50:13.237 INFO:teuthology.orchestra.run.trial160.stdout:(81/158): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2026-02-26T03:50:13.308 INFO:teuthology.orchestra.run.trial160.stdout:(82/158): python3-iniconfig-1.1.1-7.el9.noarch. 34 kB/s | 17 kB 00:00 2026-02-26T03:50:13.515 INFO:teuthology.orchestra.run.trial160.stdout:(83/158): python3-devel-3.9.25-3.el9.x86_64.rpm 309 kB/s | 244 kB 00:00 2026-02-26T03:50:13.926 INFO:teuthology.orchestra.run.trial160.stdout:(84/158): python3-jmespath-1.0.1-1.el9.noarch.r 77 kB/s | 48 kB 00:00 2026-02-26T03:50:14.052 INFO:teuthology.orchestra.run.trial160.stdout:(85/158): python3-jinja2-2.11.3-8.el9.noarch.rp 305 kB/s | 249 kB 00:00 2026-02-26T03:50:14.269 INFO:teuthology.orchestra.run.trial160.stdout:(86/158): python3-libstoragemgmt-1.10.1-1.el9.x 235 kB/s | 177 kB 00:00 2026-02-26T03:50:14.634 INFO:teuthology.orchestra.run.trial160.stdout:(87/158): python3-markupsafe-1.1.1-12.el9.x86_6 60 kB/s | 35 kB 00:00 2026-02-26T03:50:14.903 INFO:teuthology.orchestra.run.trial160.stdout:(88/158): python3-lxml-4.6.5-3.el9.x86_64.rpm 1.3 MB/s | 1.2 MB 00:00 2026-02-26T03:50:15.438 INFO:teuthology.orchestra.run.trial160.stdout:(89/158): python3-numpy-1.23.5-2.el9.x86_64.rpm 5.2 MB/s | 6.1 MB 00:01 2026-02-26T03:50:15.521 INFO:teuthology.orchestra.run.trial160.stdout:(90/158): python3-numpy-f2py-1.23.5-2.el9.x86_6 499 kB/s | 442 kB 00:00 2026-02-26T03:50:15.575 INFO:teuthology.orchestra.run.trial160.stdout:(91/158): python3-packaging-20.9-5.el9.noarch.r 115 kB/s | 77 kB 00:00 2026-02-26T03:50:15.581 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:50:15.581 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-2:20.3.0-5312.gabffb617.el9.x86_64 1/160 2026-02-26T03:50:15.582 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 2/160 2026-02-26T03:50:15.582 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 3/160 2026-02-26T03:50:15.582 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86 4/160 2026-02-26T03:50:15.582 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:50:15.582 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5312.gabffb 6/160 2026-02-26T03:50:15.582 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 7/160 2026-02-26T03:50:15.582 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 8/160 2026-02-26T03:50:15.582 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 9/160 2026-02-26T03:50:15.582 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 10/160 2026-02-26T03:50:15.582 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 11/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 12/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb6 13/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 14/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 15/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86 17/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_ 18/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 19/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 20/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 21/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librados-devel-2:20.3.0-5312.gabffb617.el9.x86_6 22/160 2026-02-26T03:50:15.583 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 23/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 24/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5312.gabffb617.el 25/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-ceph-common-2:20.3.0-5312.gabffb617.el9. 26/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_6 27/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 28/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 29/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 30/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 31/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 32/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 33/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5312.gabffb617. 34/160 2026-02-26T03:50:15.584 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 35/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 36/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 37/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 38/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el 39/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 40/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.e 41/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 42/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : cephadm-2:20.3.0-5312.gabffb617.el9.noarch 43/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 44/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 45/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 46/160 2026-02-26T03:50:15.585 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 47/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 48/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 49/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 50/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 51/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 52/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 53/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 54/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 55/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 56/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-ply-3.11-14.el9.noarch 57/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 58/160 2026-02-26T03:50:15.586 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 59/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 60/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 61/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : unzip-6.0-59.el9.x86_64 62/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : zip-3.0-35.el9.x86_64 63/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 64/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 65/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 66/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 67/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 68/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 69/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 70/160 2026-02-26T03:50:15.587 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 71/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 72/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 73/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 74/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 75/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 76/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : lua-5.4.4-4.el9.x86_64 77/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 78/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 79/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 80/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 81/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 82/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 83/160 2026-02-26T03:50:15.588 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 84/160 2026-02-26T03:50:15.589 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 85/160 2026-02-26T03:50:15.589 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 86/160 2026-02-26T03:50:15.589 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 87/160 2026-02-26T03:50:15.589 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 88/160 2026-02-26T03:50:15.589 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 89/160 2026-02-26T03:50:15.589 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 90/160 2026-02-26T03:50:15.589 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 91/160 2026-02-26T03:50:15.589 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 92/160 2026-02-26T03:50:15.589 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 93/160 2026-02-26T03:50:15.589 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 94/160 2026-02-26T03:50:15.589 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 95/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 96/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pytest-6.2.2-7.el9.noarch 97/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 98/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 99/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 100/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 101/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 102/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 103/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 104/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 105/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 106/160 2026-02-26T03:50:15.590 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 107/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 108/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 109/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 110/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 111/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 112/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 113/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 114/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 115/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 116/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 117/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 118/160 2026-02-26T03:50:15.591 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 119/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 120/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 121/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 122/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 123/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 124/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 125/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 126/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 127/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 128/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 129/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 130/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 131/160 2026-02-26T03:50:15.592 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 132/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 133/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 134/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 135/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 136/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 137/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 138/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 139/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 140/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 141/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 142/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 143/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 144/160 2026-02-26T03:50:15.593 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 145/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 146/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 147/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 148/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 149/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 150/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 151/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 152/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 153/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : re2-1:20211101-20.el9.x86_64 154/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 155/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 156/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librados2-2:20.3.0-5312.gabffb617.el9.x86_64 157/160 2026-02-26T03:50:15.594 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 158/160 2026-02-26T03:50:15.595 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 159/160 2026-02-26T03:50:15.654 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:50:15.654 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:50:15.654 INFO:teuthology.orchestra.run.trial195.stdout:Upgraded: 2026-02-26T03:50:15.654 INFO:teuthology.orchestra.run.trial195.stdout: librados2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.655 INFO:teuthology.orchestra.run.trial195.stdout: librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.655 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-26T03:50:15.655 INFO:teuthology.orchestra.run.trial195.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-26T03:50:15.655 INFO:teuthology.orchestra.run.trial195.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-26T03:50:15.655 INFO:teuthology.orchestra.run.trial195.stdout: ceph-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.655 INFO:teuthology.orchestra.run.trial195.stdout: ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.655 INFO:teuthology.orchestra.run.trial195.stdout: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.655 INFO:teuthology.orchestra.run.trial195.stdout: ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.655 INFO:teuthology.orchestra.run.trial195.stdout: ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.655 INFO:teuthology.orchestra.run.trial195.stdout: ceph-grafana-dashboards-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:50:15.655 INFO:teuthology.orchestra.run.trial195.stdout: ceph-immutable-object-cache-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.656 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.656 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.656 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:50:15.656 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:50:15.656 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:50:15.656 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:50:15.656 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:50:15.656 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:50:15.656 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.656 INFO:teuthology.orchestra.run.trial195.stdout: ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: cephadm-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-26T03:50:15.657 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.658 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: librados-devel-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-26T03:50:15.659 INFO:teuthology.orchestra.run.trial195.stdout: librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: lua-5.4.4-4.el9.x86_64 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-26T03:50:15.660 INFO:teuthology.orchestra.run.trial195.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: python3-ceph-argparse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.661 INFO:teuthology.orchestra.run.trial195.stdout: python3-ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2026-02-26T03:50:15.662 INFO:teuthology.orchestra.run.trial195.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-26T03:50:15.663 INFO:teuthology.orchestra.run.trial195.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-pluggy-0.13.1-7.el9.noarch 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-ply-3.11-14.el9.noarch 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-py-1.10.0-6.el9.noarch 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-26T03:50:15.664 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-pytest-6.2.2-7.el9.noarch 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-26T03:50:15.665 INFO:teuthology.orchestra.run.trial195.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-26T03:50:15.666 INFO:teuthology.orchestra.run.trial195.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: re2-1:20211101-20.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: unzip-6.0-59.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: zip-3.0-35.el9.x86_64 2026-02-26T03:50:15.667 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:50:15.668 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-26T03:50:15.805 DEBUG:teuthology.parallel:result is None 2026-02-26T03:50:16.036 INFO:teuthology.orchestra.run.trial160.stdout:(92/158): python3-pluggy-0.13.1-7.el9.noarch.rp 69 kB/s | 41 kB 00:00 2026-02-26T03:50:16.343 INFO:teuthology.orchestra.run.trial160.stdout:(93/158): python3-protobuf-3.14.0-17.el9.noarch 326 kB/s | 267 kB 00:00 2026-02-26T03:50:16.439 INFO:teuthology.orchestra.run.trial160.stdout:(94/158): python3-py-1.10.0-6.el9.noarch.rpm 550 kB/s | 477 kB 00:00 2026-02-26T03:50:16.773 INFO:teuthology.orchestra.run.trial160.stdout:(95/158): python3-pyasn1-0.4.8-7.el9.noarch.rpm 214 kB/s | 157 kB 00:00 2026-02-26T03:50:17.146 INFO:teuthology.orchestra.run.trial160.stdout:(96/158): python3-pyasn1-modules-0.4.8-7.el9.no 345 kB/s | 277 kB 00:00 2026-02-26T03:50:17.340 INFO:teuthology.orchestra.run.trial160.stdout:(97/158): python3-pytest-6.2.2-7.el9.noarch.rpm 576 kB/s | 519 kB 00:00 2026-02-26T03:50:17.393 INFO:teuthology.orchestra.run.trial160.stdout:(98/158): python3-requests-oauthlib-1.3.0-12.el 86 kB/s | 54 kB 00:00 2026-02-26T03:50:17.933 INFO:teuthology.orchestra.run.trial160.stdout:(99/158): python3-toml-0.10.2-6.el9.noarch.rpm 70 kB/s | 42 kB 00:00 2026-02-26T03:50:18.209 INFO:teuthology.orchestra.run.trial160.stdout:(100/158): qatlib-25.08.0-2.el9.x86_64.rpm 294 kB/s | 240 kB 00:00 2026-02-26T03:50:18.540 INFO:teuthology.orchestra.run.trial160.stdout:(101/158): qatlib-service-25.08.0-2.el9.x86_64. 61 kB/s | 37 kB 00:00 2026-02-26T03:50:18.660 INFO:teuthology.orchestra.run.trial160.stdout:(102/158): python3-scipy-1.9.3-2.el9.x86_64.rpm 13 MB/s | 19 MB 00:01 2026-02-26T03:50:18.854 INFO:teuthology.orchestra.run.trial160.stdout:(103/158): qatzip-libs-1.3.1-1.el9.x86_64.rpm 103 kB/s | 66 kB 00:00 2026-02-26T03:50:19.354 INFO:teuthology.orchestra.run.trial160.stdout:(104/158): socat-1.7.4.1-8.el9.x86_64.rpm 373 kB/s | 303 kB 00:00 2026-02-26T03:50:19.414 INFO:teuthology.orchestra.run.trial160.stdout:(105/158): xmlsec1-1.2.29-13.el9.x86_64.rpm 250 kB/s | 189 kB 00:00 2026-02-26T03:50:19.541 INFO:teuthology.orchestra.run.trial160.stdout:(106/158): xmlsec1-openssl-1.2.29-13.el9.x86_64 132 kB/s | 90 kB 00:00 2026-02-26T03:50:19.995 INFO:teuthology.orchestra.run.trial160.stdout:(107/158): xmlstarlet-1.6.1-20.el9.x86_64.rpm 99 kB/s | 64 kB 00:00 2026-02-26T03:50:20.186 INFO:teuthology.orchestra.run.trial160.stdout:(108/158): abseil-cpp-20211102.0-4.el9.x86_64.r 2.8 MB/s | 551 kB 00:00 2026-02-26T03:50:20.218 INFO:teuthology.orchestra.run.trial160.stdout:(109/158): gperftools-libs-2.9.1-3.el9.x86_64.r 9.4 MB/s | 308 kB 00:00 2026-02-26T03:50:20.242 INFO:teuthology.orchestra.run.trial160.stdout:(110/158): grpc-data-1.46.7-10.el9.noarch.rpm 808 kB/s | 19 kB 00:00 2026-02-26T03:50:20.348 INFO:teuthology.orchestra.run.trial160.stdout:(111/158): libarrow-9.0.0-15.el9.x86_64.rpm 42 MB/s | 4.4 MB 00:00 2026-02-26T03:50:20.372 INFO:teuthology.orchestra.run.trial160.stdout:(112/158): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.0 MB/s | 25 kB 00:00 2026-02-26T03:50:20.398 INFO:teuthology.orchestra.run.trial160.stdout:(113/158): liboath-2.6.12-1.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2026-02-26T03:50:20.423 INFO:teuthology.orchestra.run.trial160.stdout:(114/158): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2026-02-26T03:50:20.449 INFO:teuthology.orchestra.run.trial160.stdout:(115/158): luarocks-3.9.2-5.el9.noarch.rpm 5.7 MB/s | 151 kB 00:00 2026-02-26T03:50:20.480 INFO:teuthology.orchestra.run.trial160.stdout:(116/158): parquet-libs-9.0.0-15.el9.x86_64.rpm 26 MB/s | 838 kB 00:00 2026-02-26T03:50:20.507 INFO:teuthology.orchestra.run.trial160.stdout:(117/158): python3-asyncssh-2.13.2-5.el9.noarch 20 MB/s | 548 kB 00:00 2026-02-26T03:50:20.530 INFO:teuthology.orchestra.run.trial160.stdout:(118/158): python3-autocommand-2.2.2-8.el9.noar 1.2 MB/s | 29 kB 00:00 2026-02-26T03:50:20.554 INFO:teuthology.orchestra.run.trial160.stdout:(119/158): python3-backports-tarfile-1.2.0-1.el 2.5 MB/s | 60 kB 00:00 2026-02-26T03:50:20.578 INFO:teuthology.orchestra.run.trial160.stdout:(120/158): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2026-02-26T03:50:20.602 INFO:teuthology.orchestra.run.trial160.stdout:(121/158): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2026-02-26T03:50:20.626 INFO:teuthology.orchestra.run.trial160.stdout:(122/158): python3-certifi-2023.05.07-4.el9.noa 607 kB/s | 14 kB 00:00 2026-02-26T03:50:20.651 INFO:teuthology.orchestra.run.trial160.stdout:(123/158): python3-cheroot-10.0.1-4.el9.noarch. 6.8 MB/s | 173 kB 00:00 2026-02-26T03:50:20.677 INFO:teuthology.orchestra.run.trial160.stdout:(124/158): python3-cherrypy-18.6.1-2.el9.noarch 14 MB/s | 358 kB 00:00 2026-02-26T03:50:20.702 INFO:teuthology.orchestra.run.trial160.stdout:(125/158): python3-google-auth-2.45.0-1.el9.noa 9.8 MB/s | 254 kB 00:00 2026-02-26T03:50:20.736 INFO:teuthology.orchestra.run.trial160.stdout:(126/158): python3-grpcio-1.46.7-10.el9.x86_64. 60 MB/s | 2.0 MB 00:00 2026-02-26T03:50:20.761 INFO:teuthology.orchestra.run.trial160.stdout:(127/158): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2026-02-26T03:50:20.786 INFO:teuthology.orchestra.run.trial160.stdout:(128/158): python3-influxdb-5.3.1-1.el9.noarch. 5.5 MB/s | 139 kB 00:00 2026-02-26T03:50:20.810 INFO:teuthology.orchestra.run.trial160.stdout:(129/158): python3-isodate-0.6.1-3.el9.noarch.r 2.3 MB/s | 56 kB 00:00 2026-02-26T03:50:20.833 INFO:teuthology.orchestra.run.trial160.stdout:(130/158): python3-jaraco-8.2.1-3.el9.noarch.rp 457 kB/s | 11 kB 00:00 2026-02-26T03:50:20.857 INFO:teuthology.orchestra.run.trial160.stdout:(131/158): python3-jaraco-classes-3.2.1-5.el9.n 755 kB/s | 18 kB 00:00 2026-02-26T03:50:20.880 INFO:teuthology.orchestra.run.trial160.stdout:(132/158): python3-jaraco-collections-3.0.0-8.e 989 kB/s | 23 kB 00:00 2026-02-26T03:50:20.904 INFO:teuthology.orchestra.run.trial160.stdout:(133/158): python3-jaraco-context-6.0.1-3.el9.n 838 kB/s | 20 kB 00:00 2026-02-26T03:50:20.928 INFO:teuthology.orchestra.run.trial160.stdout:(134/158): python3-jaraco-functools-3.5.0-2.el9 828 kB/s | 19 kB 00:00 2026-02-26T03:50:20.951 INFO:teuthology.orchestra.run.trial160.stdout:(135/158): python3-jaraco-text-4.0.0-2.el9.noar 1.1 MB/s | 26 kB 00:00 2026-02-26T03:50:20.981 INFO:teuthology.orchestra.run.trial160.stdout:(136/158): python3-kubernetes-26.1.0-3.el9.noar 35 MB/s | 1.0 MB 00:00 2026-02-26T03:50:21.004 INFO:teuthology.orchestra.run.trial160.stdout:(137/158): python3-more-itertools-8.12.0-2.el9. 3.2 MB/s | 79 kB 00:00 2026-02-26T03:50:21.029 INFO:teuthology.orchestra.run.trial160.stdout:(138/158): python3-msgpack-1.0.3-2.el9.x86_64.r 3.5 MB/s | 86 kB 00:00 2026-02-26T03:50:21.053 INFO:teuthology.orchestra.run.trial160.stdout:(139/158): python3-natsort-7.1.1-5.el9.noarch.r 2.4 MB/s | 58 kB 00:00 2026-02-26T03:50:21.076 INFO:teuthology.orchestra.run.trial160.stdout:(140/158): python3-portend-3.1.0-2.el9.noarch.r 703 kB/s | 16 kB 00:00 2026-02-26T03:50:21.101 INFO:teuthology.orchestra.run.trial160.stdout:(141/158): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2026-02-26T03:50:21.124 INFO:teuthology.orchestra.run.trial160.stdout:(142/158): python3-repoze-lru-0.7-16.el9.noarch 1.3 MB/s | 31 kB 00:00 2026-02-26T03:50:21.149 INFO:teuthology.orchestra.run.trial160.stdout:(143/158): python3-routes-2.5.1-5.el9.noarch.rp 7.3 MB/s | 188 kB 00:00 2026-02-26T03:50:21.173 INFO:teuthology.orchestra.run.trial160.stdout:(144/158): python3-rsa-4.9-2.el9.noarch.rpm 2.4 MB/s | 59 kB 00:00 2026-02-26T03:50:21.198 INFO:teuthology.orchestra.run.trial160.stdout:(145/158): python3-saml-1.16.0-1.el9.noarch.rpm 5.0 MB/s | 125 kB 00:00 2026-02-26T03:50:21.222 INFO:teuthology.orchestra.run.trial160.stdout:(146/158): python3-tempora-5.0.0-2.el9.noarch.r 1.5 MB/s | 36 kB 00:00 2026-02-26T03:50:21.246 INFO:teuthology.orchestra.run.trial160.stdout:(147/158): python3-typing-extensions-4.15.0-1.e 3.4 MB/s | 86 kB 00:00 2026-02-26T03:50:21.270 INFO:teuthology.orchestra.run.trial160.stdout:(148/158): python3-websocket-client-1.2.3-2.el9 3.6 MB/s | 90 kB 00:00 2026-02-26T03:50:21.294 INFO:teuthology.orchestra.run.trial160.stdout:(149/158): python3-xmlsec-1.3.13-1.el9.x86_64.r 2.0 MB/s | 48 kB 00:00 2026-02-26T03:50:21.318 INFO:teuthology.orchestra.run.trial160.stdout:(150/158): python3-xmltodict-0.12.0-15.el9.noar 944 kB/s | 22 kB 00:00 2026-02-26T03:50:21.342 INFO:teuthology.orchestra.run.trial160.stdout:(151/158): python3-zc-lockfile-2.0-10.el9.noarc 853 kB/s | 20 kB 00:00 2026-02-26T03:50:21.367 INFO:teuthology.orchestra.run.trial160.stdout:(152/158): re2-20211101-20.el9.x86_64.rpm 7.4 MB/s | 191 kB 00:00 2026-02-26T03:50:21.399 INFO:teuthology.orchestra.run.trial160.stdout:(153/158): thrift-0.15.0-4.el9.x86_64.rpm 50 MB/s | 1.6 MB 00:00 2026-02-26T03:50:21.423 INFO:teuthology.orchestra.run.trial160.stdout:(154/158): yaml-cpp-0.6.3-5.el9.x86_64.rpm 4.9 MB/s | 122 kB 00:00 2026-02-26T03:50:22.008 INFO:teuthology.orchestra.run.trial160.stdout:(155/158): librados2-20.3.0-5312.gabffb617.el9. 5.6 MB/s | 3.3 MB 00:00 2026-02-26T03:50:22.382 INFO:teuthology.orchestra.run.trial160.stdout:(156/158): librbd1-20.3.0-5312.gabffb617.el9.x8 7.2 MB/s | 2.7 MB 00:00 2026-02-26T03:50:36.203 INFO:teuthology.orchestra.run.trial160.stdout:(157/158): lua-devel-5.4.4-4.el9.x86_64.rpm 1.3 kB/s | 22 kB 00:16 2026-02-26T03:50:36.556 INFO:teuthology.orchestra.run.trial160.stdout:(158/158): protobuf-compiler-3.14.0-17.el9.x86_ 51 kB/s | 862 kB 00:17 2026-02-26T03:50:36.557 INFO:teuthology.orchestra.run.trial160.stdout:-------------------------------------------------------------------------------- 2026-02-26T03:50:36.557 INFO:teuthology.orchestra.run.trial160.stdout:Total 14 MB/s | 863 MB 01:03 2026-02-26T03:50:37.108 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction check 2026-02-26T03:50:37.161 INFO:teuthology.orchestra.run.trial160.stdout:Transaction check succeeded. 2026-02-26T03:50:37.161 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction test 2026-02-26T03:50:39.906 INFO:teuthology.orchestra.run.trial160.stdout:Transaction test succeeded. 2026-02-26T03:50:39.906 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction 2026-02-26T03:50:42.712 INFO:teuthology.orchestra.run.trial160.stdout: Preparing : 1/1 2026-02-26T03:50:42.715 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/160 2026-02-26T03:50:42.723 INFO:teuthology.orchestra.run.trial160.stdout: Installing : liboath-2.6.12-1.el9.x86_64 2/160 2026-02-26T03:50:42.736 INFO:teuthology.orchestra.run.trial160.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/160 2026-02-26T03:50:42.879 INFO:teuthology.orchestra.run.trial160.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2026-02-26T03:50:42.880 INFO:teuthology.orchestra.run.trial160.stdout: Upgrading : librados2-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:50:42.929 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: librados2-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:50:42.929 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 6/160 2026-02-26T03:50:42.941 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 6/160 2026-02-26T03:50:42.943 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86 7/160 2026-02-26T03:50:42.943 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 8/160 2026-02-26T03:50:42.971 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 8/160 2026-02-26T03:50:42.977 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 9/160 2026-02-26T03:50:42.985 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 10/160 2026-02-26T03:50:42.987 INFO:teuthology.orchestra.run.trial160.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 11/160 2026-02-26T03:50:42.990 INFO:teuthology.orchestra.run.trial160.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 12/160 2026-02-26T03:50:42.992 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 13/160 2026-02-26T03:50:42.997 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 14/160 2026-02-26T03:50:43.114 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 15/160 2026-02-26T03:50:43.115 INFO:teuthology.orchestra.run.trial160.stdout: Upgrading : librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:50:43.158 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:50:43.164 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 17/160 2026-02-26T03:50:43.172 INFO:teuthology.orchestra.run.trial160.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 18/160 2026-02-26T03:50:43.173 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 19/160 2026-02-26T03:50:43.196 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 19/160 2026-02-26T03:50:43.196 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 20/160 2026-02-26T03:50:43.209 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 20/160 2026-02-26T03:50:43.213 INFO:teuthology.orchestra.run.trial160.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 21/160 2026-02-26T03:50:43.243 INFO:teuthology.orchestra.run.trial160.stdout: Installing : re2-1:20211101-20.el9.x86_64 22/160 2026-02-26T03:50:43.248 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 23/160 2026-02-26T03:50:43.252 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 24/160 2026-02-26T03:50:43.273 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 25/160 2026-02-26T03:50:43.279 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 26/160 2026-02-26T03:50:43.282 INFO:teuthology.orchestra.run.trial160.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2026-02-26T03:50:43.287 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 28/160 2026-02-26T03:50:43.287 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 29/160 2026-02-26T03:50:43.300 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 29/160 2026-02-26T03:50:43.305 INFO:teuthology.orchestra.run.trial160.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 30/160 2026-02-26T03:50:43.328 INFO:teuthology.orchestra.run.trial160.stdout: Installing : unzip-6.0-59.el9.x86_64 31/160 2026-02-26T03:50:43.341 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 32/160 2026-02-26T03:50:43.346 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 33/160 2026-02-26T03:50:43.349 INFO:teuthology.orchestra.run.trial160.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 34/160 2026-02-26T03:50:43.355 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 35/160 2026-02-26T03:50:43.357 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 36/160 2026-02-26T03:50:43.358 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 37/160 2026-02-26T03:50:43.368 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 38/160 2026-02-26T03:50:43.403 INFO:teuthology.orchestra.run.trial160.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 39/160 2026-02-26T03:50:43.407 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-ceph-common-2:20.3.0-5312.gabffb617.el9. 40/160 2026-02-26T03:50:43.418 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-ceph-argparse-2:20.3.0-5312.gabffb617.el 41/160 2026-02-26T03:50:43.429 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_6 42/160 2026-02-26T03:50:43.435 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 43/160 2026-02-26T03:50:43.459 INFO:teuthology.orchestra.run.trial160.stdout: Installing : zip-3.0-35.el9.x86_64 44/160 2026-02-26T03:50:43.463 INFO:teuthology.orchestra.run.trial160.stdout: Installing : luarocks-3.9.2-5.el9.noarch 45/160 2026-02-26T03:50:43.513 INFO:teuthology.orchestra.run.trial160.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 46/160 2026-02-26T03:50:43.526 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 47/160 2026-02-26T03:50:43.533 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-rsa-4.9-2.el9.noarch 48/160 2026-02-26T03:50:43.536 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/160 2026-02-26T03:50:43.538 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/160 2026-02-26T03:50:43.542 INFO:teuthology.orchestra.run.trial160.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 51/160 2026-02-26T03:50:43.559 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 52/160 2026-02-26T03:50:43.564 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 53/160 2026-02-26T03:50:43.568 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/160 2026-02-26T03:50:43.575 INFO:teuthology.orchestra.run.trial160.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 55/160 2026-02-26T03:50:43.582 INFO:teuthology.orchestra.run.trial160.stdout: Installing : librados-devel-2:20.3.0-5312.gabffb617.el9.x86_6 56/160 2026-02-26T03:50:43.594 INFO:teuthology.orchestra.run.trial160.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 57/160 2026-02-26T03:50:43.599 INFO:teuthology.orchestra.run.trial160.stdout: Installing : thrift-0.15.0-4.el9.x86_64 58/160 2026-02-26T03:50:43.602 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 59/160 2026-02-26T03:50:43.616 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 60/160 2026-02-26T03:50:43.621 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 61/160 2026-02-26T03:50:43.626 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 62/160 2026-02-26T03:50:43.637 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 63/160 2026-02-26T03:50:43.646 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 64/160 2026-02-26T03:50:43.652 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 65/160 2026-02-26T03:50:43.672 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 66/160 2026-02-26T03:50:43.683 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 67/160 2026-02-26T03:50:43.698 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 68/160 2026-02-26T03:50:43.703 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 69/160 2026-02-26T03:50:43.710 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 70/160 2026-02-26T03:50:43.750 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 71/160 2026-02-26T03:50:44.074 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 72/160 2026-02-26T03:50:44.084 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 73/160 2026-02-26T03:50:44.088 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 74/160 2026-02-26T03:50:44.093 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 75/160 2026-02-26T03:50:44.099 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 76/160 2026-02-26T03:50:44.103 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 77/160 2026-02-26T03:50:44.106 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/160 2026-02-26T03:50:44.112 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 79/160 2026-02-26T03:50:44.114 INFO:teuthology.orchestra.run.trial160.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 80/160 2026-02-26T03:50:44.116 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 81/160 2026-02-26T03:50:44.144 INFO:teuthology.orchestra.run.trial160.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 82/160 2026-02-26T03:50:44.187 INFO:teuthology.orchestra.run.trial160.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 83/160 2026-02-26T03:50:44.196 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 84/160 2026-02-26T03:50:44.243 INFO:teuthology.orchestra.run.trial160.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 85/160 2026-02-26T03:50:44.272 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-py-1.10.0-6.el9.noarch 86/160 2026-02-26T03:50:44.281 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 87/160 2026-02-26T03:50:44.288 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 88/160 2026-02-26T03:50:44.294 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 89/160 2026-02-26T03:50:44.301 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 90/160 2026-02-26T03:50:44.305 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 91/160 2026-02-26T03:50:44.339 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 92/160 2026-02-26T03:50:44.561 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 93/160 2026-02-26T03:50:44.585 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 94/160 2026-02-26T03:50:44.589 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 95/160 2026-02-26T03:50:44.647 INFO:teuthology.orchestra.run.trial160.stdout: Installing : openblas-0.3.29-1.el9.x86_64 96/160 2026-02-26T03:50:44.649 INFO:teuthology.orchestra.run.trial160.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 97/160 2026-02-26T03:50:44.670 INFO:teuthology.orchestra.run.trial160.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 98/160 2026-02-26T03:50:44.994 INFO:teuthology.orchestra.run.trial160.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 99/160 2026-02-26T03:50:45.066 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 100/160 2026-02-26T03:50:45.735 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 101/160 2026-02-26T03:50:45.749 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 102/160 2026-02-26T03:50:45.755 INFO:teuthology.orchestra.run.trial160.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 103/160 2026-02-26T03:50:46.005 INFO:teuthology.orchestra.run.trial160.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 104/160 2026-02-26T03:50:46.006 INFO:teuthology.orchestra.run.trial160.stdout: Installing : librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 105/160 2026-02-26T03:50:46.024 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 105/160 2026-02-26T03:50:46.026 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 106/160 2026-02-26T03:50:46.978 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:50:46.981 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:50:47.000 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 107/160 2026-02-26T03:50:47.011 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 108/160 2026-02-26T03:50:47.019 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-packaging-20.9-5.el9.noarch 109/160 2026-02-26T03:50:47.034 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-ply-3.11-14.el9.noarch 110/160 2026-02-26T03:50:47.050 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 111/160 2026-02-26T03:50:47.125 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 112/160 2026-02-26T03:50:47.134 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 113/160 2026-02-26T03:50:47.157 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 114/160 2026-02-26T03:50:47.187 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 115/160 2026-02-26T03:50:47.237 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 116/160 2026-02-26T03:50:47.243 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 117/160 2026-02-26T03:50:47.247 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 118/160 2026-02-26T03:50:47.252 INFO:teuthology.orchestra.run.trial160.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 119/160 2026-02-26T03:50:47.254 INFO:teuthology.orchestra.run.trial160.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 120/160 2026-02-26T03:50:47.255 INFO:teuthology.orchestra.run.trial160.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 121/160 2026-02-26T03:50:47.271 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 121/160 2026-02-26T03:50:47.385 INFO:teuthology.orchestra.run.trial160.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 122/160 2026-02-26T03:50:47.387 INFO:teuthology.orchestra.run.trial160.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 123/160 2026-02-26T03:50:47.669 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 123/160 2026-02-26T03:50:47.669 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-26T03:50:47.669 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:50:47.928 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 124/160 2026-02-26T03:50:47.965 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 124/160 2026-02-26T03:50:47.966 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-26T03:50:47.966 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-26T03:50:47.966 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:50:47.969 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 125/160 2026-02-26T03:50:52.698 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 125/160 2026-02-26T03:50:52.699 INFO:teuthology.orchestra.run.trial160.stdout:skipping the directory /sys 2026-02-26T03:50:52.699 INFO:teuthology.orchestra.run.trial160.stdout:skipping the directory /proc 2026-02-26T03:50:52.699 INFO:teuthology.orchestra.run.trial160.stdout:skipping the directory /mnt 2026-02-26T03:50:52.699 INFO:teuthology.orchestra.run.trial160.stdout:skipping the directory /var/tmp 2026-02-26T03:50:52.699 INFO:teuthology.orchestra.run.trial160.stdout:skipping the directory /home 2026-02-26T03:50:52.699 INFO:teuthology.orchestra.run.trial160.stdout:skipping the directory /root 2026-02-26T03:50:52.699 INFO:teuthology.orchestra.run.trial160.stdout:skipping the directory /tmp 2026-02-26T03:50:52.699 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:50:52.789 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 126/160 2026-02-26T03:50:52.812 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 126/160 2026-02-26T03:50:52.812 INFO:teuthology.orchestra.run.trial160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:50:52.812 INFO:teuthology.orchestra.run.trial160.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-26T03:50:52.812 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-26T03:50:52.812 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-26T03:50:52.813 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:50:53.022 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 127/160 2026-02-26T03:50:53.044 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 127/160 2026-02-26T03:50:53.044 INFO:teuthology.orchestra.run.trial160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:50:53.044 INFO:teuthology.orchestra.run.trial160.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-26T03:50:53.044 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-26T03:50:53.044 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-26T03:50:53.044 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:50:53.050 INFO:teuthology.orchestra.run.trial160.stdout: Installing : mailcap-2.1.49-5.el9.noarch 128/160 2026-02-26T03:50:53.052 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 129/160 2026-02-26T03:50:53.070 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:50:53.070 INFO:teuthology.orchestra.run.trial160.stdout:Creating group 'qat' with GID 992. 2026-02-26T03:50:53.070 INFO:teuthology.orchestra.run.trial160.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-26T03:50:53.070 INFO:teuthology.orchestra.run.trial160.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-26T03:50:53.070 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:50:53.078 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:50:53.102 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-26T03:50:53.102 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-26T03:50:53.102 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:50:53.137 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 131/160 2026-02-26T03:50:53.732 INFO:teuthology.orchestra.run.trial160.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 132/160 2026-02-26T03:50:59.133 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 133/160 2026-02-26T03:50:59.196 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 134/160 2026-02-26T03:50:59.198 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 135/160 2026-02-26T03:50:59.212 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 135/160 2026-02-26T03:50:59.212 INFO:teuthology.orchestra.run.trial160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:50:59.212 INFO:teuthology.orchestra.run.trial160.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-26T03:50:59.213 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:50:59.223 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 136/160 2026-02-26T03:50:59.246 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 136/160 2026-02-26T03:50:59.246 INFO:teuthology.orchestra.run.trial160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:50:59.246 INFO:teuthology.orchestra.run.trial160.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-26T03:50:59.246 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-26T03:50:59.246 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-26T03:50:59.246 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:50:59.305 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: cephadm-2:20.3.0-5312.gabffb617.el9.noarch 137/160 2026-02-26T03:50:59.307 INFO:teuthology.orchestra.run.trial160.stdout: Installing : cephadm-2:20.3.0-5312.gabffb617.el9.noarch 137/160 2026-02-26T03:50:59.316 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.e 138/160 2026-02-26T03:50:59.340 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5312.gabffb617. 139/160 2026-02-26T03:50:59.341 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 140/160 2026-02-26T03:51:02.263 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 140/160 2026-02-26T03:51:02.270 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 141/160 2026-02-26T03:51:02.725 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 141/160 2026-02-26T03:51:02.726 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 142/160 2026-02-26T03:51:02.738 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 142/160 2026-02-26T03:51:02.739 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 143/160 2026-02-26T03:51:02.800 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 143/160 2026-02-26T03:51:02.841 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el 144/160 2026-02-26T03:51:02.842 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 145/160 2026-02-26T03:51:02.863 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 145/160 2026-02-26T03:51:02.863 INFO:teuthology.orchestra.run.trial160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:51:02.863 INFO:teuthology.orchestra.run.trial160.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-26T03:51:02.863 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-26T03:51:02.863 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-26T03:51:02.863 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:51:02.876 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 146/160 2026-02-26T03:51:05.142 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 146/160 2026-02-26T03:51:27.563 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86 147/160 2026-02-26T03:51:27.564 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb6 148/160 2026-02-26T03:51:28.053 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-2:20.3.0-5312.gabffb617.el9.x86_64 149/160 2026-02-26T03:51:28.055 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 150/160 2026-02-26T03:51:28.077 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 150/160 2026-02-26T03:51:28.077 INFO:teuthology.orchestra.run.trial160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:51:28.077 INFO:teuthology.orchestra.run.trial160.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-26T03:51:28.078 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-26T03:51:28.078 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-26T03:51:28.078 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:51:28.086 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5312.gabffb 151/160 2026-02-26T03:51:28.107 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5312.gabffb 151/160 2026-02-26T03:51:28.107 INFO:teuthology.orchestra.run.trial160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:51:28.107 INFO:teuthology.orchestra.run.trial160.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-26T03:51:28.107 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:51:28.219 INFO:teuthology.orchestra.run.trial160.stdout: Installing : rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 152/160 2026-02-26T03:51:28.240 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 152/160 2026-02-26T03:51:28.240 INFO:teuthology.orchestra.run.trial160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-26T03:51:28.240 INFO:teuthology.orchestra.run.trial160.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-26T03:51:28.240 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-26T03:51:28.241 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-26T03:51:28.241 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:51:35.260 INFO:teuthology.orchestra.run.trial160.stdout: Installing : python3-pytest-6.2.2-7.el9.noarch 153/160 2026-02-26T03:51:35.268 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 154/160 2026-02-26T03:51:35.273 INFO:teuthology.orchestra.run.trial160.stdout: Installing : libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_ 155/160 2026-02-26T03:51:35.282 INFO:teuthology.orchestra.run.trial160.stdout: Installing : rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 156/160 2026-02-26T03:51:35.323 INFO:teuthology.orchestra.run.trial160.stdout: Installing : rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 157/160 2026-02-26T03:51:35.326 INFO:teuthology.orchestra.run.trial160.stdout: Installing : ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 158/160 2026-02-26T03:51:35.326 INFO:teuthology.orchestra.run.trial160.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/160 2026-02-26T03:51:35.339 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/160 2026-02-26T03:51:35.339 INFO:teuthology.orchestra.run.trial160.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:51:35.350 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:51:35.356 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 160/160 2026-02-26T03:51:35.379 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 160/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-2:20.3.0-5312.gabffb617.el9.x86_64 1/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 2/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 3/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86 4/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 5/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5312.gabffb 6/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 7/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 8/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 9/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 10/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86 11/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86 12/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb6 13/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 14/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 15/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 16/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86 17/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_ 18/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86 19/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 20/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 21/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : librados-devel-2:20.3.0-5312.gabffb617.el9.x86_6 22/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86 23/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 24/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5312.gabffb617.el 25/160 2026-02-26T03:51:38.405 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-ceph-common-2:20.3.0-5312.gabffb617.el9. 26/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_6 27/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 28/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 29/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 30/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 31/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 32/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 33/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5312.gabffb617. 34/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noa 35/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.n 36/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5312.gabf 37/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.n 38/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el 39/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 40/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.e 41/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 42/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : cephadm-2:20.3.0-5312.gabffb617.el9.noarch 43/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 44/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 45/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 46/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 47/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 48/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 49/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 50/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 51/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 52/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 53/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 54/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 55/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 56/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-ply-3.11-14.el9.noarch 57/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 58/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 59/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 60/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 61/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : unzip-6.0-59.el9.x86_64 62/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : zip-3.0-35.el9.x86_64 63/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 64/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 65/160 2026-02-26T03:51:38.406 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 66/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 67/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 68/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 69/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 70/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 71/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 72/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 73/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 74/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 75/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 76/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : lua-5.4.4-4.el9.x86_64 77/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 78/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 79/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 80/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 81/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 82/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 83/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 84/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 85/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 86/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 87/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 88/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 89/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 90/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 91/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 92/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 93/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 94/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 95/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 96/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-pytest-6.2.2-7.el9.noarch 97/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 98/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 99/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 100/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 101/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 102/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 103/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 104/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 105/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 106/160 2026-02-26T03:51:38.407 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 107/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 108/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 109/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 110/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 111/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 112/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 113/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 114/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 115/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 116/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 117/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 118/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 119/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 120/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 121/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 122/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 123/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 124/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 125/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 126/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 127/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 128/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 129/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 130/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 131/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 132/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 133/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 134/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 135/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 136/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 137/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 138/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 139/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 140/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 141/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 142/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 143/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 144/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 145/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 146/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 147/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 148/160 2026-02-26T03:51:38.408 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 149/160 2026-02-26T03:51:38.409 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 150/160 2026-02-26T03:51:38.409 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 151/160 2026-02-26T03:51:38.409 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 152/160 2026-02-26T03:51:38.409 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 153/160 2026-02-26T03:51:38.409 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : re2-1:20211101-20.el9.x86_64 154/160 2026-02-26T03:51:38.409 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 155/160 2026-02-26T03:51:38.409 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 156/160 2026-02-26T03:51:38.409 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : librados2-2:20.3.0-5312.gabffb617.el9.x86_64 157/160 2026-02-26T03:51:38.409 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 158/160 2026-02-26T03:51:38.409 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 159/160 2026-02-26T03:51:38.480 INFO:teuthology.orchestra.run.trial160.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 160/160 2026-02-26T03:51:38.480 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:51:38.480 INFO:teuthology.orchestra.run.trial160.stdout:Upgraded: 2026-02-26T03:51:38.480 INFO:teuthology.orchestra.run.trial160.stdout: librados2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.480 INFO:teuthology.orchestra.run.trial160.stdout: librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.480 INFO:teuthology.orchestra.run.trial160.stdout:Installed: 2026-02-26T03:51:38.480 INFO:teuthology.orchestra.run.trial160.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-26T03:51:38.480 INFO:teuthology.orchestra.run.trial160.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-26T03:51:38.480 INFO:teuthology.orchestra.run.trial160.stdout: ceph-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-debugsource-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-grafana-dashboards-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-immutable-object-cache-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mds-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:51:38.481 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-k8sevents-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-modules-core-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-mon-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-osd-crimson-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-prometheus-alerts-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-selinux-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.482 INFO:teuthology.orchestra.run.trial160.stdout: ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: cephadm-2:20.3.0-5312.gabffb617.el9.noarch 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-26T03:51:38.483 INFO:teuthology.orchestra.run.trial160.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: libcephfs-daemon-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: libcephfs-proxy2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: libcephsqlite-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-26T03:51:38.484 INFO:teuthology.orchestra.run.trial160.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: librados-devel-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: libradosstriper1-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: librgw2-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: lua-5.4.4-4.el9.x86_64 2026-02-26T03:51:38.485 INFO:teuthology.orchestra.run.trial160.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-26T03:51:38.486 INFO:teuthology.orchestra.run.trial160.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-26T03:51:38.487 INFO:teuthology.orchestra.run.trial160.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-26T03:51:38.487 INFO:teuthology.orchestra.run.trial160.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-26T03:51:38.487 INFO:teuthology.orchestra.run.trial160.stdout: python3-ceph-argparse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.487 INFO:teuthology.orchestra.run.trial160.stdout: python3-ceph-common-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.487 INFO:teuthology.orchestra.run.trial160.stdout: python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.487 INFO:teuthology.orchestra.run.trial160.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-26T03:51:38.487 INFO:teuthology.orchestra.run.trial160.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-26T03:51:38.487 INFO:teuthology.orchestra.run.trial160.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-26T03:51:38.487 INFO:teuthology.orchestra.run.trial160.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-26T03:51:38.487 INFO:teuthology.orchestra.run.trial160.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-26T03:51:38.488 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-26T03:51:38.489 INFO:teuthology.orchestra.run.trial160.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-pluggy-0.13.1-7.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-ply-3.11-14.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-py-1.10.0-6.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-pytest-6.2.2-7.el9.noarch 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.490 INFO:teuthology.orchestra.run.trial160.stdout: python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-26T03:51:38.491 INFO:teuthology.orchestra.run.trial160.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: re2-1:20211101-20.el9.x86_64 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-26T03:51:38.492 INFO:teuthology.orchestra.run.trial160.stdout: unzip-6.0-59.el9.x86_64 2026-02-26T03:51:38.493 INFO:teuthology.orchestra.run.trial160.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-26T03:51:38.493 INFO:teuthology.orchestra.run.trial160.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-26T03:51:38.493 INFO:teuthology.orchestra.run.trial160.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-26T03:51:38.493 INFO:teuthology.orchestra.run.trial160.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-26T03:51:38.493 INFO:teuthology.orchestra.run.trial160.stdout: zip-3.0-35.el9.x86_64 2026-02-26T03:51:38.493 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:51:38.493 INFO:teuthology.orchestra.run.trial160.stdout:Complete! 2026-02-26T03:51:38.630 DEBUG:teuthology.parallel:result is None 2026-02-26T03:51:38.631 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:51:38.805 DEBUG:teuthology.orchestra.run.trial160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-26T03:51:38.828 INFO:teuthology.orchestra.run.trial160.stdout:20.3.0-5312.gabffb617.el9 2026-02-26T03:51:38.829 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5312.gabffb617.el9 2026-02-26T03:51:38.829 INFO:teuthology.task.install:The correct ceph version 20.3.0-5312.gabffb617 is installed. 2026-02-26T03:51:38.831 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:51:38.979 DEBUG:teuthology.orchestra.run.trial195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-26T03:51:39.014 INFO:teuthology.orchestra.run.trial195.stdout:20.3.0-5312.gabffb617.el9 2026-02-26T03:51:39.014 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5312.gabffb617.el9 2026-02-26T03:51:39.014 INFO:teuthology.task.install:The correct ceph version 20.3.0-5312.gabffb617 is installed. 2026-02-26T03:51:39.018 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-26T03:51:39.018 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:51:39.018 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-26T03:51:39.047 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:51:39.047 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-26T03:51:39.081 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-26T03:51:39.081 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:51:39.081 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/usr/bin/daemon-helper 2026-02-26T03:51:39.111 DEBUG:teuthology.orchestra.run.trial160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-26T03:51:39.175 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:51:39.175 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/usr/bin/daemon-helper 2026-02-26T03:51:39.201 DEBUG:teuthology.orchestra.run.trial195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-26T03:51:39.265 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-26T03:51:39.266 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:51:39.266 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-26T03:51:39.292 DEBUG:teuthology.orchestra.run.trial160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-26T03:51:39.357 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:51:39.357 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-26T03:51:39.382 DEBUG:teuthology.orchestra.run.trial195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-26T03:51:39.446 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-26T03:51:39.446 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:51:39.446 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/usr/bin/stdin-killer 2026-02-26T03:51:39.471 DEBUG:teuthology.orchestra.run.trial160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-26T03:51:39.535 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:51:39.535 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/usr/bin/stdin-killer 2026-02-26T03:51:39.560 DEBUG:teuthology.orchestra.run.trial195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-26T03:51:39.624 INFO:teuthology.run_tasks:Running task pexec... 2026-02-26T03:51:39.643 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-26T03:51:39.644 DEBUG:teuthology.orchestra.run.trial160:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-26T03:51:39.644 DEBUG:teuthology.orchestra.run.trial195:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-26T03:51:39.647 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial160.front.sepia.ceph.com 2026-02-26T03:51:39.647 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-26T03:51:39.667 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial195.front.sepia.ceph.com 2026-02-26T03:51:39.667 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-26T03:51:39.697 DEBUG:teuthology.parallel:result is None 2026-02-26T03:51:39.716 DEBUG:teuthology.parallel:result is None 2026-02-26T03:51:39.716 INFO:teuthology.run_tasks:Running task install... 2026-02-26T03:51:39.727 DEBUG:teuthology.task.install:project ceph 2026-02-26T03:51:39.728 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'}} 2026-02-26T03:51:39.728 DEBUG:teuthology.task.install:config {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'} 2026-02-26T03:51:39.728 INFO:teuthology.task.install:Using flavor: debug 2026-02-26T03:51:39.731 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-26T03:51:39.731 INFO:teuthology.task.install:extra packages: [] 2026-02-26T03:51:39.731 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'tag': None, 'wait_for_package': False} 2026-02-26T03:51:39.731 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:51:39.732 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'tag': None, 'wait_for_package': False} 2026-02-26T03:51:39.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:51:39.896 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/ 2026-02-26T03:51:39.896 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5312.gabffb617 2026-02-26T03:51:39.907 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/ 2026-02-26T03:51:39.907 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5312.gabffb617 2026-02-26T03:51:40.239 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-26T03:51:40.240 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:51:40.240 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-26T03:51:40.264 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest on remote rpm x86_64 2026-02-26T03:51:40.265 DEBUG:teuthology.orchestra.run.trial160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-26T03:51:40.286 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-peering-bug/abffb617683951511fccf3c3cab6e219f2ec83a8/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-26T03:51:40.286 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:51:40.286 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-26T03:51:40.312 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest on remote rpm x86_64 2026-02-26T03:51:40.312 DEBUG:teuthology.orchestra.run.trial195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-26T03:51:40.330 DEBUG:teuthology.orchestra.run.trial160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-26T03:51:40.376 DEBUG:teuthology.orchestra.run.trial195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-26T03:51:40.395 DEBUG:teuthology.orchestra.run.trial160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-26T03:51:40.440 DEBUG:teuthology.orchestra.run.trial195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-26T03:51:40.451 INFO:teuthology.orchestra.run.trial160.stdout:check_obsoletes = 1 2026-02-26T03:51:40.461 DEBUG:teuthology.orchestra.run.trial160:> sudo yum clean all 2026-02-26T03:51:40.496 INFO:teuthology.orchestra.run.trial195.stdout:check_obsoletes = 1 2026-02-26T03:51:40.506 DEBUG:teuthology.orchestra.run.trial195:> sudo yum clean all 2026-02-26T03:51:40.627 INFO:teuthology.orchestra.run.trial160.stdout:56 files removed 2026-02-26T03:51:40.643 DEBUG:teuthology.orchestra.run.trial160:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest 2026-02-26T03:51:40.667 INFO:teuthology.orchestra.run.trial195.stdout:56 files removed 2026-02-26T03:51:40.682 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest 2026-02-26T03:51:41.481 INFO:teuthology.orchestra.run.trial160.stdout:ceph packages for x86_64 138 kB/s | 94 kB 00:00 2026-02-26T03:51:41.528 INFO:teuthology.orchestra.run.trial195.stdout:ceph packages for x86_64 136 kB/s | 94 kB 00:00 2026-02-26T03:51:42.179 INFO:teuthology.orchestra.run.trial195.stdout:ceph noarch packages 39 kB/s | 24 kB 00:00 2026-02-26T03:51:42.194 INFO:teuthology.orchestra.run.trial160.stdout:ceph noarch packages 36 kB/s | 24 kB 00:00 2026-02-26T03:51:42.703 INFO:teuthology.orchestra.run.trial195.stdout:ceph source packages 4.1 kB/s | 2.1 kB 00:00 2026-02-26T03:51:42.723 INFO:teuthology.orchestra.run.trial160.stdout:ceph source packages 4.1 kB/s | 2.1 kB 00:00 2026-02-26T03:51:43.382 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.9 MB 00:00 2026-02-26T03:51:43.594 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.9 MB 00:00 2026-02-26T03:51:46.844 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 27 MB 00:02 2026-02-26T03:51:46.894 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 27 MB 00:02 2026-02-26T03:51:50.799 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 8.0 MB 00:01 2026-02-26T03:51:51.021 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 8.0 MB 00:01 2026-02-26T03:51:51.846 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - Extras packages 256 kB/s | 20 kB 00:00 2026-02-26T03:51:52.363 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 20 kB 00:00 2026-02-26T03:51:53.005 INFO:teuthology.orchestra.run.trial195.stdout:Extra Packages for Enterprise Linux 35 MB/s | 20 MB 00:00 2026-02-26T03:51:53.843 INFO:teuthology.orchestra.run.trial160.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2026-02-26T03:51:56.195 INFO:teuthology.orchestra.run.trial195.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-26T03:51:57.082 INFO:teuthology.orchestra.run.trial160.stdout:lab-extras 2.4 MB/s | 50 kB 00:00 2026-02-26T03:51:57.389 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.390 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.390 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.391 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.391 INFO:teuthology.orchestra.run.trial195.stdout:Package cephadm-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:57.392 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-immutable-object-cache-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.392 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.393 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:57.393 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:57.393 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:57.394 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:57.394 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.395 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.395 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.395 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:57.396 INFO:teuthology.orchestra.run.trial195.stdout:Package librados-devel-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.396 INFO:teuthology.orchestra.run.trial195.stdout:Package libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.397 INFO:teuthology.orchestra.run.trial195.stdout:Package libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.397 INFO:teuthology.orchestra.run.trial195.stdout:Package librados2-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.398 INFO:teuthology.orchestra.run.trial195.stdout:Package librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.398 INFO:teuthology.orchestra.run.trial195.stdout:Package python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.399 INFO:teuthology.orchestra.run.trial195.stdout:Package python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.399 INFO:teuthology.orchestra.run.trial195.stdout:Package python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.399 INFO:teuthology.orchestra.run.trial195.stdout:Package python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.400 INFO:teuthology.orchestra.run.trial195.stdout:Package rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.400 INFO:teuthology.orchestra.run.trial195.stdout:Package rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.401 INFO:teuthology.orchestra.run.trial195.stdout:Package rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:57.401 INFO:teuthology.orchestra.run.trial195.stdout:Package python3-pytest-6.2.2-7.el9.noarch is already installed. 2026-02-26T03:51:57.419 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-26T03:51:57.420 INFO:teuthology.orchestra.run.trial195.stdout:Nothing to do. 2026-02-26T03:51:57.420 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-26T03:51:57.448 DEBUG:teuthology.parallel:result is None 2026-02-26T03:51:58.288 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-radosgw-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.288 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-test-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.289 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.289 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-base-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.289 INFO:teuthology.orchestra.run.trial160.stdout:Package cephadm-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:58.290 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-immutable-object-cache-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.290 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-mgr-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.290 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-mgr-dashboard-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:58.291 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:58.291 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-mgr-rook-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:58.292 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-mgr-cephadm-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:58.292 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-osd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.293 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-osd-classic-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.293 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.294 INFO:teuthology.orchestra.run.trial160.stdout:Package ceph-volume-2:20.3.0-5312.gabffb617.el9.noarch is already installed. 2026-02-26T03:51:58.294 INFO:teuthology.orchestra.run.trial160.stdout:Package librados-devel-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.294 INFO:teuthology.orchestra.run.trial160.stdout:Package libcephfs2-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.295 INFO:teuthology.orchestra.run.trial160.stdout:Package libcephfs-devel-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.295 INFO:teuthology.orchestra.run.trial160.stdout:Package librados2-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.296 INFO:teuthology.orchestra.run.trial160.stdout:Package librbd1-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.296 INFO:teuthology.orchestra.run.trial160.stdout:Package python3-rados-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.297 INFO:teuthology.orchestra.run.trial160.stdout:Package python3-rgw-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.297 INFO:teuthology.orchestra.run.trial160.stdout:Package python3-cephfs-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.298 INFO:teuthology.orchestra.run.trial160.stdout:Package python3-rbd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.298 INFO:teuthology.orchestra.run.trial160.stdout:Package rbd-fuse-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.299 INFO:teuthology.orchestra.run.trial160.stdout:Package rbd-mirror-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.299 INFO:teuthology.orchestra.run.trial160.stdout:Package rbd-nbd-2:20.3.0-5312.gabffb617.el9.x86_64 is already installed. 2026-02-26T03:51:58.299 INFO:teuthology.orchestra.run.trial160.stdout:Package python3-pytest-6.2.2-7.el9.noarch is already installed. 2026-02-26T03:51:58.318 INFO:teuthology.orchestra.run.trial160.stdout:Dependencies resolved. 2026-02-26T03:51:58.319 INFO:teuthology.orchestra.run.trial160.stdout:Nothing to do. 2026-02-26T03:51:58.319 INFO:teuthology.orchestra.run.trial160.stdout:Complete! 2026-02-26T03:51:58.349 DEBUG:teuthology.parallel:result is None 2026-02-26T03:51:58.350 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:51:58.518 DEBUG:teuthology.orchestra.run.trial160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-26T03:51:58.540 INFO:teuthology.orchestra.run.trial160.stdout:20.3.0-5312.gabffb617.el9 2026-02-26T03:51:58.540 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5312.gabffb617.el9 2026-02-26T03:51:58.540 INFO:teuthology.task.install:The correct ceph version 20.3.0-5312.gabffb617 is installed. 2026-02-26T03:51:58.543 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=abffb617683951511fccf3c3cab6e219f2ec83a8 2026-02-26T03:51:58.685 DEBUG:teuthology.orchestra.run.trial195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-26T03:51:58.705 INFO:teuthology.orchestra.run.trial195.stdout:20.3.0-5312.gabffb617.el9 2026-02-26T03:51:58.705 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5312.gabffb617.el9 2026-02-26T03:51:58.706 INFO:teuthology.task.install:The correct ceph version 20.3.0-5312.gabffb617 is installed. 2026-02-26T03:51:58.707 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-26T03:51:58.707 INFO:teuthology.run_tasks:Running task ceph... 2026-02-26T03:51:58.785 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-26T03:51:58.785 DEBUG:teuthology.orchestra.run.trial160:> sudo chmod 777 /var/log/ceph 2026-02-26T03:51:58.787 DEBUG:teuthology.orchestra.run.trial195:> sudo chmod 777 /var/log/ceph 2026-02-26T03:51:58.811 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-26T03:51:58.811 DEBUG:teuthology.orchestra.run.trial160:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-26T03:51:58.853 DEBUG:teuthology.orchestra.run.trial195:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-26T03:51:58.876 INFO:tasks.ceph:Creating extra log directories... 2026-02-26T03:51:58.876 DEBUG:teuthology.orchestra.run.trial160:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-26T03:51:58.917 DEBUG:teuthology.orchestra.run.trial195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-26T03:51:58.951 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-26T03:51:58.951 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-26T03:51:58.951 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71262', 'branch': 'main', 'description': 'crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/rados_python}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '71262', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996'}}, 'owner': 'scheduled_shraddhaag@soko04.front.sepia.ceph.com', 'priority': 101, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5117, 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996', 'targets': {'trial160.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAmKoMZxpPU5T/Bxq+g9wCBUqaaDDrIIcYb15cINtnZ1/IWb/NZ2PmjEABs3NWcoKUjOhdh2PYGMEcMdXyBJrLU=', 'trial195.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC9C7nCXyYm8D8zvsOtcO6RiBr8aoxp5NEqzbJfdx5dtzsZdhTqOM5G0vbA7nwTDtdiSgpuy/b6p7GvF1bjCxDU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'abffb617683951511fccf3c3cab6e219f2ec83a8', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ["rados/test_python.sh -m 'not (wait or tier or ec)'"]}, 'timeout': '1h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '6e3d0419ccca0e75a03e08feb708946e198236f4', 'timestamp': '2026-02-26_02:22:56', 'tube': 'trial', 'user': 'shraddhaag', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699'} 2026-02-26T03:51:58.952 DEBUG:teuthology.orchestra.run.trial160:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-26T03:51:58.982 DEBUG:teuthology.orchestra.run.trial195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-26T03:51:58.997 DEBUG:teuthology.orchestra.run.trial160:> sudo install -d -m0777 -- /var/run/ceph 2026-02-26T03:51:59.038 DEBUG:teuthology.orchestra.run.trial195:> sudo install -d -m0777 -- /var/run/ceph 2026-02-26T03:51:59.062 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:51:59.063 DEBUG:teuthology.orchestra.run.trial160:> dd if=/scratch_devs of=/dev/stdout 2026-02-26T03:51:59.117 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-26T03:51:59.117 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/vg_nvme/lv_1 2026-02-26T03:51:59.173 INFO:teuthology.orchestra.run.trial160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-26T03:51:59.173 INFO:teuthology.orchestra.run.trial160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:51:59.173 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d Inode: 1308 Links: 1 2026-02-26T03:51:59.173 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:51:59.173 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:51:59.173 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-26 03:51:35.578873639 +0000 2026-02-26T03:51:59.173 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-26 03:50:47.694880131 +0000 2026-02-26T03:51:59.173 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-26 03:50:47.694880131 +0000 2026-02-26T03:51:59.173 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-26 03:50:47.694880131 +0000 2026-02-26T03:51:59.173 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-26T03:51:59.235 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in 2026-02-26T03:51:59.236 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out 2026-02-26T03:51:59.236 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000162814 s, 3.1 MB/s 2026-02-26T03:51:59.237 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-26T03:51:59.294 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/vg_nvme/lv_2 2026-02-26T03:51:59.350 INFO:teuthology.orchestra.run.trial160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-26T03:51:59.351 INFO:teuthology.orchestra.run.trial160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:51:59.351 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d Inode: 1320 Links: 1 2026-02-26T03:51:59.351 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:51:59.351 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:51:59.351 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-26 03:51:35.579873642 +0000 2026-02-26T03:51:59.351 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-26 03:50:47.696880091 +0000 2026-02-26T03:51:59.351 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-26 03:50:47.696880091 +0000 2026-02-26T03:51:59.351 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-26 03:50:47.696880091 +0000 2026-02-26T03:51:59.352 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-26T03:51:59.412 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in 2026-02-26T03:51:59.413 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out 2026-02-26T03:51:59.413 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000146704 s, 3.5 MB/s 2026-02-26T03:51:59.414 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-26T03:51:59.471 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/vg_nvme/lv_3 2026-02-26T03:51:59.527 INFO:teuthology.orchestra.run.trial160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-26T03:51:59.527 INFO:teuthology.orchestra.run.trial160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:51:59.528 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d Inode: 1301 Links: 1 2026-02-26T03:51:59.528 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:51:59.528 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:51:59.528 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-26 03:51:35.579873642 +0000 2026-02-26T03:51:59.528 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-26 03:50:47.694880131 +0000 2026-02-26T03:51:59.528 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-26 03:50:47.694880131 +0000 2026-02-26T03:51:59.528 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-26 03:50:47.694880131 +0000 2026-02-26T03:51:59.528 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-26T03:51:59.589 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in 2026-02-26T03:51:59.589 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out 2026-02-26T03:51:59.589 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000153106 s, 3.3 MB/s 2026-02-26T03:51:59.590 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-26T03:51:59.647 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/vg_nvme/lv_4 2026-02-26T03:51:59.703 INFO:teuthology.orchestra.run.trial160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-26T03:51:59.703 INFO:teuthology.orchestra.run.trial160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:51:59.704 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d Inode: 1292 Links: 1 2026-02-26T03:51:59.704 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:51:59.704 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:51:59.704 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-26 03:51:35.579873642 +0000 2026-02-26T03:51:59.704 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-26 03:50:47.693880152 +0000 2026-02-26T03:51:59.704 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-26 03:50:47.693880152 +0000 2026-02-26T03:51:59.704 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-26 03:50:47.693880152 +0000 2026-02-26T03:51:59.704 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-26T03:51:59.768 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in 2026-02-26T03:51:59.768 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out 2026-02-26T03:51:59.768 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000142826 s, 3.6 MB/s 2026-02-26T03:51:59.769 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-26T03:51:59.825 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-26T03:51:59.825 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:51:59.825 DEBUG:teuthology.orchestra.run.trial195:> dd if=/scratch_devs of=/dev/stdout 2026-02-26T03:51:59.844 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-26T03:51:59.845 DEBUG:teuthology.orchestra.run.trial195:> stat /dev/vg_nvme/lv_1 2026-02-26T03:51:59.902 INFO:teuthology.orchestra.run.trial195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-26T03:51:59.902 INFO:teuthology.orchestra.run.trial195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:51:59.902 INFO:teuthology.orchestra.run.trial195.stdout:Device: 6h/6d Inode: 1358 Links: 1 2026-02-26T03:51:59.902 INFO:teuthology.orchestra.run.trial195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:51:59.902 INFO:teuthology.orchestra.run.trial195.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:51:59.902 INFO:teuthology.orchestra.run.trial195.stdout:Access: 2026-02-26 03:50:12.776055323 +0000 2026-02-26T03:51:59.902 INFO:teuthology.orchestra.run.trial195.stdout:Modify: 2026-02-26 03:49:25.539823009 +0000 2026-02-26T03:51:59.902 INFO:teuthology.orchestra.run.trial195.stdout:Change: 2026-02-26 03:49:25.539823009 +0000 2026-02-26T03:51:59.903 INFO:teuthology.orchestra.run.trial195.stdout: Birth: 2026-02-26 03:49:25.539823009 +0000 2026-02-26T03:51:59.903 DEBUG:teuthology.orchestra.run.trial195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-26T03:51:59.966 INFO:teuthology.orchestra.run.trial195.stderr:1+0 records in 2026-02-26T03:51:59.966 INFO:teuthology.orchestra.run.trial195.stderr:1+0 records out 2026-02-26T03:51:59.967 INFO:teuthology.orchestra.run.trial195.stderr:512 bytes copied, 0.000230939 s, 2.2 MB/s 2026-02-26T03:51:59.968 DEBUG:teuthology.orchestra.run.trial195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-26T03:52:00.025 DEBUG:teuthology.orchestra.run.trial195:> stat /dev/vg_nvme/lv_2 2026-02-26T03:52:00.081 INFO:teuthology.orchestra.run.trial195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-26T03:52:00.082 INFO:teuthology.orchestra.run.trial195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:52:00.082 INFO:teuthology.orchestra.run.trial195.stdout:Device: 6h/6d Inode: 1348 Links: 1 2026-02-26T03:52:00.082 INFO:teuthology.orchestra.run.trial195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:52:00.082 INFO:teuthology.orchestra.run.trial195.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:52:00.082 INFO:teuthology.orchestra.run.trial195.stdout:Access: 2026-02-26 03:50:12.776055323 +0000 2026-02-26T03:52:00.082 INFO:teuthology.orchestra.run.trial195.stdout:Modify: 2026-02-26 03:49:25.538823020 +0000 2026-02-26T03:52:00.082 INFO:teuthology.orchestra.run.trial195.stdout:Change: 2026-02-26 03:49:25.538823020 +0000 2026-02-26T03:52:00.082 INFO:teuthology.orchestra.run.trial195.stdout: Birth: 2026-02-26 03:49:25.538823020 +0000 2026-02-26T03:52:00.083 DEBUG:teuthology.orchestra.run.trial195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-26T03:52:00.144 INFO:teuthology.orchestra.run.trial195.stderr:1+0 records in 2026-02-26T03:52:00.144 INFO:teuthology.orchestra.run.trial195.stderr:1+0 records out 2026-02-26T03:52:00.144 INFO:teuthology.orchestra.run.trial195.stderr:512 bytes copied, 0.000228595 s, 2.2 MB/s 2026-02-26T03:52:00.146 DEBUG:teuthology.orchestra.run.trial195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-26T03:52:00.203 DEBUG:teuthology.orchestra.run.trial195:> stat /dev/vg_nvme/lv_3 2026-02-26T03:52:00.259 INFO:teuthology.orchestra.run.trial195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-26T03:52:00.259 INFO:teuthology.orchestra.run.trial195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:52:00.260 INFO:teuthology.orchestra.run.trial195.stdout:Device: 6h/6d Inode: 1343 Links: 1 2026-02-26T03:52:00.260 INFO:teuthology.orchestra.run.trial195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:52:00.260 INFO:teuthology.orchestra.run.trial195.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:52:00.260 INFO:teuthology.orchestra.run.trial195.stdout:Access: 2026-02-26 03:50:12.776055323 +0000 2026-02-26T03:52:00.260 INFO:teuthology.orchestra.run.trial195.stdout:Modify: 2026-02-26 03:49:25.538823020 +0000 2026-02-26T03:52:00.260 INFO:teuthology.orchestra.run.trial195.stdout:Change: 2026-02-26 03:49:25.538823020 +0000 2026-02-26T03:52:00.260 INFO:teuthology.orchestra.run.trial195.stdout: Birth: 2026-02-26 03:49:25.538823020 +0000 2026-02-26T03:52:00.260 DEBUG:teuthology.orchestra.run.trial195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-26T03:52:00.322 INFO:teuthology.orchestra.run.trial195.stderr:1+0 records in 2026-02-26T03:52:00.322 INFO:teuthology.orchestra.run.trial195.stderr:1+0 records out 2026-02-26T03:52:00.322 INFO:teuthology.orchestra.run.trial195.stderr:512 bytes copied, 0.00015033 s, 3.4 MB/s 2026-02-26T03:52:00.324 DEBUG:teuthology.orchestra.run.trial195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-26T03:52:00.380 DEBUG:teuthology.orchestra.run.trial195:> stat /dev/vg_nvme/lv_4 2026-02-26T03:52:00.437 INFO:teuthology.orchestra.run.trial195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-26T03:52:00.437 INFO:teuthology.orchestra.run.trial195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-26T03:52:00.438 INFO:teuthology.orchestra.run.trial195.stdout:Device: 6h/6d Inode: 1349 Links: 1 2026-02-26T03:52:00.438 INFO:teuthology.orchestra.run.trial195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-26T03:52:00.438 INFO:teuthology.orchestra.run.trial195.stdout:Context: system_u:object_r:device_t:s0 2026-02-26T03:52:00.438 INFO:teuthology.orchestra.run.trial195.stdout:Access: 2026-02-26 03:50:12.777055302 +0000 2026-02-26T03:52:00.438 INFO:teuthology.orchestra.run.trial195.stdout:Modify: 2026-02-26 03:49:25.538823020 +0000 2026-02-26T03:52:00.438 INFO:teuthology.orchestra.run.trial195.stdout:Change: 2026-02-26 03:49:25.538823020 +0000 2026-02-26T03:52:00.438 INFO:teuthology.orchestra.run.trial195.stdout: Birth: 2026-02-26 03:49:25.538823020 +0000 2026-02-26T03:52:00.439 DEBUG:teuthology.orchestra.run.trial195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-26T03:52:00.501 INFO:teuthology.orchestra.run.trial195.stderr:1+0 records in 2026-02-26T03:52:00.501 INFO:teuthology.orchestra.run.trial195.stderr:1+0 records out 2026-02-26T03:52:00.501 INFO:teuthology.orchestra.run.trial195.stderr:512 bytes copied, 0.000165047 s, 3.1 MB/s 2026-02-26T03:52:00.502 DEBUG:teuthology.orchestra.run.trial195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-26T03:52:00.559 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-26T03:52:00.559 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial160.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial195.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-26T03:52:00.559 INFO:tasks.ceph:Generating config... 2026-02-26T03:52:00.560 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-26T03:52:00.560 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-26T03:52:00.560 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-26T03:52:00.560 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-26T03:52:00.560 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-26T03:52:00.560 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-26T03:52:00.560 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-26T03:52:00.560 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-26T03:52:00.560 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-26T03:52:00.560 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-26T03:52:00.561 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-26T03:52:00.561 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-26T03:52:00.561 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-26T03:52:00.561 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-26T03:52:00.561 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-26T03:52:00.561 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-26T03:52:00.561 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-26T03:52:00.561 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-26T03:52:00.561 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-26T03:52:00.561 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-26T03:52:00.561 INFO:tasks.ceph:Setting up mon.a... 2026-02-26T03:52:00.561 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-26T03:52:00.603 INFO:teuthology.orchestra.run.trial160.stdout:creating /etc/ceph/ceph.keyring 2026-02-26T03:52:00.606 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-26T03:52:00.690 DEBUG:teuthology.orchestra.run.trial160:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-26T03:52:00.754 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.160')] 2026-02-26T03:52:00.755 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.160', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-26T03:52:00.757 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:52:00.757 DEBUG:teuthology.orchestra.run.trial160:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-26T03:52:00.810 DEBUG:teuthology.orchestra.run.trial160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.160 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:00.885+0000 7f2677e84f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:00.885+0000 7f2677e84f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:00.886+0000 7f2677e84f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stdout:monmaptool: generated fsid 2350f431-1ca3-4096-a5f6-1c0f74b874c1 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stdout:setting min_mon_release = tentacle 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stdout:epoch 0 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stdout:fsid 2350f431-1ca3-4096-a5f6-1c0f74b874c1 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stdout:last_changed 2026-02-26T03:52:00.887638+0000 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stdout:created 2026-02-26T03:52:00.887638+0000 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stdout:min_mon_release 20 (tentacle) 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stdout:election_strategy: 1 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stdout:0: [v2:10.20.193.160:3300/0,v1:10.20.193.160:6789/0] mon.a 2026-02-26T03:52:00.889 INFO:teuthology.orchestra.run.trial160.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-26T03:52:00.892 DEBUG:teuthology.orchestra.run.trial160:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-26T03:52:00.947 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2350f431-1ca3-4096-a5f6-1c0f74b874c1... 2026-02-26T03:52:00.947 DEBUG:teuthology.orchestra.run.trial160:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-26T03:52:00.989 DEBUG:teuthology.orchestra.run.trial195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-26T03:52:01.027 INFO:teuthology.orchestra.run.trial160.stdout:[global] 2026-02-26T03:52:01.028 INFO:teuthology.orchestra.run.trial160.stdout: chdir = "" 2026-02-26T03:52:01.028 INFO:teuthology.orchestra.run.trial160.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-26T03:52:01.028 INFO:teuthology.orchestra.run.trial160.stdout: auth supported = cephx 2026-02-26T03:52:01.028 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.028 INFO:teuthology.orchestra.run.trial160.stdout: filestore xattr use omap = true 2026-02-26T03:52:01.028 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.028 INFO:teuthology.orchestra.run.trial160.stdout: mon clock drift allowed = 1.000 2026-02-26T03:52:01.028 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.028 INFO:teuthology.orchestra.run.trial160.stdout: osd crush chooseleaf type = 0 2026-02-26T03:52:01.029 INFO:teuthology.orchestra.run.trial160.stdout: auth debug = true 2026-02-26T03:52:01.029 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.029 INFO:teuthology.orchestra.run.trial160.stdout: ms die on old message = true 2026-02-26T03:52:01.029 INFO:teuthology.orchestra.run.trial160.stdout: ms die on bug = true 2026-02-26T03:52:01.029 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.029 INFO:teuthology.orchestra.run.trial160.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-26T03:52:01.029 INFO:teuthology.orchestra.run.trial160.stdout: mon pg warn max object skew = 0 2026-02-26T03:52:01.029 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.029 INFO:teuthology.orchestra.run.trial160.stdout: # disable pg_autoscaler by default for new pools 2026-02-26T03:52:01.029 INFO:teuthology.orchestra.run.trial160.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-26T03:52:01.029 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: osd pool default size = 2 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: mon osd allow primary affinity = true 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: mon osd allow pg remap = true 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: mon warn on legacy crush tunables = false 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: mon warn on crush straw calc version zero = false 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: mon warn on no sortbitwise = false 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: mon warn on osd down out interval zero = false 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: mon warn on too few osds = false 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-26T03:52:01.030 INFO:teuthology.orchestra.run.trial160.stdout: mon_allow_pool_size_one = true 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: osd default data pool replay window = 5 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: mon allow pool delete = true 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: mon cluster log file level = debug 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: debug asserts on shutdown = true 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: mon health detail to clog = false 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: mon host = 10.20.193.160 2026-02-26T03:52:01.031 INFO:teuthology.orchestra.run.trial160.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-26T03:52:01.032 INFO:teuthology.orchestra.run.trial160.stdout: ms client mode = crc 2026-02-26T03:52:01.032 INFO:teuthology.orchestra.run.trial160.stdout: ms cluster mode = crc 2026-02-26T03:52:01.032 INFO:teuthology.orchestra.run.trial160.stdout: ms mon client mode = crc 2026-02-26T03:52:01.032 INFO:teuthology.orchestra.run.trial160.stdout: ms mon cluster mode = crc 2026-02-26T03:52:01.032 INFO:teuthology.orchestra.run.trial160.stdout: ms mon service mode = crc 2026-02-26T03:52:01.032 INFO:teuthology.orchestra.run.trial160.stdout: ms service mode = crc 2026-02-26T03:52:01.032 INFO:teuthology.orchestra.run.trial160.stdout: fsid = 2350f431-1ca3-4096-a5f6-1c0f74b874c1 2026-02-26T03:52:01.032 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.032 INFO:teuthology.orchestra.run.trial160.stdout:[osd] 2026-02-26T03:52:01.032 INFO:teuthology.orchestra.run.trial160.stdout: osd journal size = 100 2026-02-26T03:52:01.032 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: osd scrub load threshold = 5.0 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: osd scrub max interval = 600 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: osd mclock profile = high_recovery_ops 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: osd mclock skip benchmark = true 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: osd recover clone overlap = true 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: osd recovery max chunk = 1048576 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: osd debug shutdown = true 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: osd debug op order = true 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: osd debug verify stray on activate = true 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: osd debug trim objects = true 2026-02-26T03:52:01.033 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: osd open classes on start = true 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: osd debug pg log writeout = true 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: osd deep scrub update digest min age = 30 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: osd map max advance = 10 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: journal zero on create = true 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: filestore ondisk finisher threads = 3 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: filestore apply finisher threads = 3 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.034 INFO:teuthology.orchestra.run.trial160.stdout: bdev debug aio = true 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: osd debug misdirected ops = true 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: debug monc = 20 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: crimson cpu num = 4 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: crimson osd obc lru size = 10 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: debug ms = 20 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: debug osd = 20 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore = 20 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore backref = 20 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore cache = 20 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore cleaner = 20 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore device = 20 2026-02-26T03:52:01.035 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore epm = 20 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore fixedkv tree = 20 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore journal = 20 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore lba = 20 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore odata = 20 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore omap = 20 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore onode = 20 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore t = 20 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: debug seastore tm = 20 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: osd objectstore = seastore 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: osd shutdown pgref assert = True 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: seastore cachepin size pershard = 64M 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: seastore cachepin type = 2Q 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-26T03:52:01.036 INFO:teuthology.orchestra.run.trial160.stdout: seastore max concurrent transactions = 8 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout:[mgr] 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout: debug ms = 1 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout: debug mgr = 20 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout: debug mon = 20 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout: debug auth = 20 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout: mon reweight min pgs per osd = 4 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout: mon reweight min bytes per osd = 10 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout: mgr/telemetry/nag = false 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout:[mon] 2026-02-26T03:52:01.037 INFO:teuthology.orchestra.run.trial160.stdout: debug ms = 1 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: debug mon = 20 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: debug paxos = 20 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: debug auth = 20 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: mon data avail warn = 5 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: mon mgr mkfs grace = 240 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: mon reweight min pgs per osd = 4 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: mon osd reporter subtree level = osd 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: mon osd prime pg temp = true 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: mon reweight min bytes per osd = 10 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: auth mon ticket ttl = 660 # 11m 2026-02-26T03:52:01.038 INFO:teuthology.orchestra.run.trial160.stdout: auth service ticket ttl = 240 # 4m 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: # don't complain about insecure global_id in the test suite 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: # 1m isn't quite enough 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: mon_down_mkfs_grace = 2m 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: mon_warn_on_filestore_osds = false 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: mon min osdmap epochs = 50 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: mon osdmap full prune interval = 2 2026-02-26T03:52:01.039 INFO:teuthology.orchestra.run.trial160.stdout: mon osdmap full prune min = 15 2026-02-26T03:52:01.040 INFO:teuthology.orchestra.run.trial160.stdout: mon osdmap full prune txsize = 2 2026-02-26T03:52:01.040 INFO:teuthology.orchestra.run.trial160.stdout: paxos service trim min = 10 2026-02-26T03:52:01.040 INFO:teuthology.orchestra.run.trial160.stdout: osd pool default crimson = True 2026-02-26T03:52:01.040 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:01.040 INFO:teuthology.orchestra.run.trial160.stdout:[client] 2026-02-26T03:52:01.040 INFO:teuthology.orchestra.run.trial160.stdout: rgw cache enabled = true 2026-02-26T03:52:01.040 INFO:teuthology.orchestra.run.trial160.stdout: rgw enable ops log = true 2026-02-26T03:52:01.040 INFO:teuthology.orchestra.run.trial160.stdout: rgw enable usage log = true 2026-02-26T03:52:01.040 INFO:teuthology.orchestra.run.trial160.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-26T03:52:01.040 INFO:teuthology.orchestra.run.trial160.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-26T03:52:01.040 INFO:teuthology.orchestra.run.trial160.stdout:[mon.a] 2026-02-26T03:52:01.042 INFO:teuthology.orchestra.run.trial195.stdout:[global] 2026-02-26T03:52:01.042 INFO:teuthology.orchestra.run.trial195.stdout: chdir = "" 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: auth supported = cephx 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: filestore xattr use omap = true 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: mon clock drift allowed = 1.000 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: osd crush chooseleaf type = 0 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: auth debug = true 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: ms die on old message = true 2026-02-26T03:52:01.043 INFO:teuthology.orchestra.run.trial195.stdout: ms die on bug = true 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: mon pg warn max object skew = 0 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: # disable pg_autoscaler by default for new pools 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: osd pool default size = 2 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: mon osd allow primary affinity = true 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: mon osd allow pg remap = true 2026-02-26T03:52:01.044 INFO:teuthology.orchestra.run.trial195.stdout: mon warn on legacy crush tunables = false 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: mon warn on crush straw calc version zero = false 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: mon warn on no sortbitwise = false 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: mon warn on osd down out interval zero = false 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: mon warn on too few osds = false 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: mon_allow_pool_size_one = true 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: osd default data pool replay window = 5 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.045 INFO:teuthology.orchestra.run.trial195.stdout: mon allow pool delete = true 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: mon cluster log file level = debug 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: debug asserts on shutdown = true 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: mon health detail to clog = false 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: mon host = 10.20.193.160 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: ms client mode = crc 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: ms cluster mode = crc 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: ms mon client mode = crc 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: ms mon cluster mode = crc 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: ms mon service mode = crc 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: ms service mode = crc 2026-02-26T03:52:01.046 INFO:teuthology.orchestra.run.trial195.stdout: fsid = 2350f431-1ca3-4096-a5f6-1c0f74b874c1 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout:[osd] 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout: osd journal size = 100 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout: osd scrub load threshold = 5.0 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout: osd scrub max interval = 600 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout: osd mclock profile = high_recovery_ops 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout: osd mclock skip benchmark = true 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout: osd recover clone overlap = true 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout: osd recovery max chunk = 1048576 2026-02-26T03:52:01.047 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: osd debug shutdown = true 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: osd debug op order = true 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: osd debug verify stray on activate = true 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: osd debug trim objects = true 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: osd open classes on start = true 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: osd debug pg log writeout = true 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: osd deep scrub update digest min age = 30 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: osd map max advance = 10 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.048 INFO:teuthology.orchestra.run.trial195.stdout: journal zero on create = true 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: filestore ondisk finisher threads = 3 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: filestore apply finisher threads = 3 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: bdev debug aio = true 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: osd debug misdirected ops = true 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: debug monc = 20 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: crimson cpu num = 4 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: crimson osd obc lru size = 10 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: debug ms = 20 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: debug osd = 20 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore = 20 2026-02-26T03:52:01.049 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore backref = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore cache = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore cleaner = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore device = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore epm = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore fixedkv tree = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore journal = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore lba = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore odata = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore omap = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore onode = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore t = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: debug seastore tm = 20 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: osd objectstore = seastore 2026-02-26T03:52:01.050 INFO:teuthology.orchestra.run.trial195.stdout: osd shutdown pgref assert = True 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout: seastore cachepin size pershard = 64M 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout: seastore cachepin type = 2Q 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout: seastore max concurrent transactions = 8 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout:[mgr] 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout: debug ms = 1 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout: debug mgr = 20 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout: debug mon = 20 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout: debug auth = 20 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout: mon reweight min pgs per osd = 4 2026-02-26T03:52:01.051 INFO:teuthology.orchestra.run.trial195.stdout: mon reweight min bytes per osd = 10 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: mgr/telemetry/nag = false 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout:[mon] 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: debug ms = 1 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: debug mon = 20 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: debug paxos = 20 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: debug auth = 20 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: mon data avail warn = 5 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: mon mgr mkfs grace = 240 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: mon reweight min pgs per osd = 4 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: mon osd reporter subtree level = osd 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: mon osd prime pg temp = true 2026-02-26T03:52:01.052 INFO:teuthology.orchestra.run.trial195.stdout: mon reweight min bytes per osd = 10 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: auth mon ticket ttl = 660 # 11m 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: auth service ticket ttl = 240 # 4m 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: # don't complain about insecure global_id in the test suite 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: # 1m isn't quite enough 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: mon_down_mkfs_grace = 2m 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.053 INFO:teuthology.orchestra.run.trial195.stdout: mon_warn_on_filestore_osds = false 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: mon min osdmap epochs = 50 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: mon osdmap full prune interval = 2 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: mon osdmap full prune min = 15 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: mon osdmap full prune txsize = 2 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: paxos service trim min = 10 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: osd pool default crimson = True 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout:[client] 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: rgw cache enabled = true 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: rgw enable ops log = true 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: rgw enable usage log = true 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-26T03:52:01.054 INFO:teuthology.orchestra.run.trial195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-26T03:52:01.055 INFO:teuthology.orchestra.run.trial195.stdout:[mon.a] 2026-02-26T03:52:01.055 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-26T03:52:01.055 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-26T03:52:01.133 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-26T03:52:01.133 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:52:01.133 DEBUG:teuthology.orchestra.run.trial160:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-26T03:52:01.149 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:52:01.149 DEBUG:teuthology.orchestra.run.trial160:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-26T03:52:01.206 INFO:tasks.ceph:Sending monmap to node ubuntu@trial160.front.sepia.ceph.com 2026-02-26T03:52:01.206 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:52:01.206 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-26T03:52:01.206 DEBUG:teuthology.orchestra.run.trial160:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-26T03:52:01.279 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:52:01.280 DEBUG:teuthology.orchestra.run.trial160:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:52:01.334 INFO:tasks.ceph:Sending monmap to node ubuntu@trial195.front.sepia.ceph.com 2026-02-26T03:52:01.334 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:52:01.335 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-26T03:52:01.335 DEBUG:teuthology.orchestra.run.trial195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-26T03:52:01.368 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:52:01.369 DEBUG:teuthology.orchestra.run.trial195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:52:01.424 INFO:tasks.ceph:Setting up mon nodes... 2026-02-26T03:52:01.424 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-26T03:52:01.424 DEBUG:teuthology.orchestra.run.trial195:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-26T03:52:01.517 INFO:teuthology.orchestra.run.trial195.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-26T03:52:01.519 INFO:tasks.ceph:Setting up mds nodes... 2026-02-26T03:52:01.520 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-26T03:52:01.520 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-26T03:52:01.561 INFO:teuthology.orchestra.run.trial160.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-26T03:52:01.574 DEBUG:teuthology.orchestra.run.trial195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2026-02-26T03:52:01.615 INFO:teuthology.orchestra.run.trial195.stdout:creating /etc/ceph/ceph.client.1.keyring 2026-02-26T03:52:01.626 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-26T03:52:01.627 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial160.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial195.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-26T03:52:01.627 DEBUG:teuthology.orchestra.run.trial160:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-26T03:52:01.651 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-26T03:52:01.651 INFO:tasks.ceph:role: osd.0 2026-02-26T03:52:01.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial160.front.sepia.ceph.com 2026-02-26T03:52:01.652 DEBUG:teuthology.orchestra.run.trial160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-26T03:52:01.716 INFO:teuthology.orchestra.run.trial160.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-26T03:52:01.716 INFO:teuthology.orchestra.run.trial160.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-26T03:52:01.716 INFO:teuthology.orchestra.run.trial160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-26T03:52:01.716 INFO:teuthology.orchestra.run.trial160.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-26T03:52:01.716 INFO:teuthology.orchestra.run.trial160.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-26T03:52:01.717 INFO:teuthology.orchestra.run.trial160.stdout: = sunit=0 swidth=0 blks 2026-02-26T03:52:01.717 INFO:teuthology.orchestra.run.trial160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-26T03:52:01.717 INFO:teuthology.orchestra.run.trial160.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-26T03:52:01.717 INFO:teuthology.orchestra.run.trial160.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-26T03:52:01.717 INFO:teuthology.orchestra.run.trial160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-26T03:52:02.285 INFO:teuthology.orchestra.run.trial160.stdout:Discarding blocks...Done. 2026-02-26T03:52:02.287 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial160.front.sepia.ceph.com -o noatime 2026-02-26T03:52:02.287 DEBUG:teuthology.orchestra.run.trial160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-26T03:52:02.356 DEBUG:teuthology.orchestra.run.trial160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-26T03:52:02.424 DEBUG:teuthology.orchestra.run.trial160:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-26T03:52:02.488 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-26T03:52:02.488 INFO:tasks.ceph:role: osd.1 2026-02-26T03:52:02.488 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial160.front.sepia.ceph.com 2026-02-26T03:52:02.488 DEBUG:teuthology.orchestra.run.trial160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-26T03:52:02.552 INFO:teuthology.orchestra.run.trial160.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-26T03:52:02.553 INFO:teuthology.orchestra.run.trial160.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-26T03:52:02.553 INFO:teuthology.orchestra.run.trial160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-26T03:52:02.553 INFO:teuthology.orchestra.run.trial160.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-26T03:52:02.553 INFO:teuthology.orchestra.run.trial160.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-26T03:52:02.553 INFO:teuthology.orchestra.run.trial160.stdout: = sunit=0 swidth=0 blks 2026-02-26T03:52:02.553 INFO:teuthology.orchestra.run.trial160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-26T03:52:02.553 INFO:teuthology.orchestra.run.trial160.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-26T03:52:02.553 INFO:teuthology.orchestra.run.trial160.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-26T03:52:02.553 INFO:teuthology.orchestra.run.trial160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-26T03:52:03.111 INFO:teuthology.orchestra.run.trial160.stdout:Discarding blocks...Done. 2026-02-26T03:52:03.113 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial160.front.sepia.ceph.com -o noatime 2026-02-26T03:52:03.113 DEBUG:teuthology.orchestra.run.trial160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-26T03:52:03.182 DEBUG:teuthology.orchestra.run.trial160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-26T03:52:03.249 DEBUG:teuthology.orchestra.run.trial160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:52:03.400 INFO:teuthology.orchestra.run.trial160.stderr:INFO  osd - OSD::main: parsing early config 2026-02-26T03:52:03.456 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:03,454 seastar - Reactor backend: linux-aio 2026-02-26T03:52:03.474 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:03,472 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:03.474 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:03,472 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:03.476 INFO:teuthology.orchestra.run.trial160.stderr:==36660==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:52:03.509 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x358a92c 0x358abb9 0x358ae1f 0x358b0e7 0x358b43b 0x358b5af 0x358b9d6 0x358bbe9 0x358be6a 0x358c02c 0x358c154 0x358dc0a 0x3590593 0x35907fc 0x3590c93 0x3590daf 0x3590fe1 0x3591264 0x35915e3 0x35918f3 0x35919b1 0x3591b30 0x35923b7 0x35927f4 0x3592b1a 0x3592c3c 0x35950fe 0x3596f97 0x359724e 0x35982df 0x359839d 0x359843f 0x3598568 0x359888d 0x3598a8f 0x3598e11 0x35d0220 0x35d0aca 0x35d102a 0x35d13a2 0x35d1840 0x35d1c7f 0x3599760 0x359a797 0x359aa4e 0x359aeed 0x359b009 0x359b2bd 0x359b64f 0x359b982 0x359bcdf 0x359bf17 0x359c552 0x359c882 0x359cbe8 0x359d095 0x359d4d4 0x359d700 0x359db0f 0x359ddc6 0x359df17 0x359e1e8 0x359e573 0x359e8a6 0x359ed99 0x359f17b 0x359f307 0x359f73f 0x359faab 0x359fe10 0x35a0133 0x35a0215 0x35a04a3 0x35a067a 0x35a08e1 0x35d3c99 0x35d43e9 0x35d48d5 0x35d4c4f 0x35d4ece 0x35d4f83 0x357dba2 0x357e505 0x357eb3f 0x357dba2 0x357dfb1 0x357e268 0x357f5c9 0x35c2afd 2026-02-26T03:52:03.531 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:03,529 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-26T03:52:03.532 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:03,529 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-26T03:52:03.532 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:03,529 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-26T03:52:03.532 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:03,529 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-26T03:52:03.532 INFO:teuthology.orchestra.run.trial160.stderr:DEBUG 2026-02-26 03:52:03,529 [shard 0:main] seastar - reactor::drain 2026-02-26T03:52:05.598 INFO:teuthology.orchestra.run.trial160.stderr:----------------------------------------------------- 2026-02-26T03:52:05.599 INFO:teuthology.orchestra.run.trial160.stderr:Suppressions used: 2026-02-26T03:52:05.599 INFO:teuthology.orchestra.run.trial160.stderr: count bytes template 2026-02-26T03:52:05.599 INFO:teuthology.orchestra.run.trial160.stderr: 1 8 MallocExtension::Register 2026-02-26T03:52:05.599 INFO:teuthology.orchestra.run.trial160.stderr:----------------------------------------------------- 2026-02-26T03:52:05.599 INFO:teuthology.orchestra.run.trial160.stderr: 2026-02-26T03:52:05.617 INFO:teuthology.orchestra.run.trial160.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-26T03:52:05.617 INFO:teuthology.orchestra.run.trial160.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-26T03:52:05.659 INFO:teuthology.orchestra.run.trial160.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-26T03:52:05.660 INFO:teuthology.orchestra.run.trial160.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-26T03:52:05.667 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:05,665 seastar - Reactor backend: linux-aio 2026-02-26T03:52:05.681 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:05,679 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:05.681 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:05,679 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:05.685 INFO:teuthology.orchestra.run.trial160.stderr:==36658==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:52:05.685 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:05,683 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-26T03:52:05.745 INFO:teuthology.orchestra.run.trial160.stderr:DEBUG 2026-02-26 03:52:05,743 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-26T03:52:05.823 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc530 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9b2559b 0x9d0c53e 0x9d0c66c 0x9d3ea77 0x9d3f21a 0x9d496e7 0x78947dd 0x7bea228 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:05.824 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x35ed7a2 0x35ed86b 0x36096e8 0x9c63157 0x9c63284 0x9d23007 0x9d2319c 0x9d2352e 0x9d24b15 0x9d25e30 0x9d26250 0x9d42adb 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:05.824 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x1125428 0xf2e0d1 0x9c50a8f 0x9c50f59 0x9c511cc 0x9c52ef3 0x9c53440 0x9c5363f 0x9d38961 0x9d390bc 0x9d4334a 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:05.968 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x9c50f6e 0x9c511cc 0x9c521cc 0x9c5273a 0x9c52a5f 0x9c52bc9 0x9d32f39 0x9d45d3c 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:05.969 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc25 /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc530 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe1d59b 0x9d092e3 0x9d1a200 0x9d1a44e 0x9d1a4fa 0x9d42619 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:05.969 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc589 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe1d59b 0x9d258d6 0x9d25a92 0x9d25bfb 0x9d25d26 0x9d26250 0x9d42adb 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:05.969 INFO:teuthology.orchestra.run.trial160.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffbafcca30 0xffffffffbafd4bab 0xffffffffbafd500b 0xffffffffbac910d2 0xffffffffbb950451 0xffffffffbba00ba2 2026-02-26T03:52:08.114 INFO:teuthology.orchestra.run.trial160.stderr:----------------------------------------------------- 2026-02-26T03:52:08.115 INFO:teuthology.orchestra.run.trial160.stderr:Suppressions used: 2026-02-26T03:52:08.115 INFO:teuthology.orchestra.run.trial160.stderr: count bytes template 2026-02-26T03:52:08.115 INFO:teuthology.orchestra.run.trial160.stderr: 1 8 MallocExtension::Register 2026-02-26T03:52:08.115 INFO:teuthology.orchestra.run.trial160.stderr:----------------------------------------------------- 2026-02-26T03:52:08.115 INFO:teuthology.orchestra.run.trial160.stderr: 2026-02-26T03:52:08.144 DEBUG:teuthology.orchestra.run.trial160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-26T03:52:08.170 DEBUG:teuthology.orchestra.run.trial160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:52:08.320 INFO:teuthology.orchestra.run.trial160.stderr:INFO  osd - OSD::main: parsing early config 2026-02-26T03:52:08.376 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:08,374 seastar - Reactor backend: linux-aio 2026-02-26T03:52:08.387 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:08,385 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:08.387 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:08,385 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:08.389 INFO:teuthology.orchestra.run.trial160.stderr:==36728==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:52:08.444 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:08,442 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-26T03:52:08.444 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:08,442 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-26T03:52:08.444 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:08,442 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-26T03:52:08.444 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:08,442 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-26T03:52:08.444 INFO:teuthology.orchestra.run.trial160.stderr:DEBUG 2026-02-26 03:52:08,442 [shard 0:main] seastar - reactor::drain 2026-02-26T03:52:10.477 INFO:teuthology.orchestra.run.trial160.stderr:----------------------------------------------------- 2026-02-26T03:52:10.478 INFO:teuthology.orchestra.run.trial160.stderr:Suppressions used: 2026-02-26T03:52:10.478 INFO:teuthology.orchestra.run.trial160.stderr: count bytes template 2026-02-26T03:52:10.478 INFO:teuthology.orchestra.run.trial160.stderr: 1 8 MallocExtension::Register 2026-02-26T03:52:10.478 INFO:teuthology.orchestra.run.trial160.stderr:----------------------------------------------------- 2026-02-26T03:52:10.478 INFO:teuthology.orchestra.run.trial160.stderr: 2026-02-26T03:52:10.497 INFO:teuthology.orchestra.run.trial160.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-26T03:52:10.497 INFO:teuthology.orchestra.run.trial160.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-26T03:52:10.546 INFO:teuthology.orchestra.run.trial160.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-26T03:52:10.546 INFO:teuthology.orchestra.run.trial160.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-26T03:52:10.553 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:10,551 seastar - Reactor backend: linux-aio 2026-02-26T03:52:10.564 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:10,562 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:10.564 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:10,562 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:10.567 INFO:teuthology.orchestra.run.trial160.stderr:==36726==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:52:10.567 INFO:teuthology.orchestra.run.trial160.stderr:INFO 2026-02-26 03:52:10,566 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-26T03:52:10.625 INFO:teuthology.orchestra.run.trial160.stderr:DEBUG 2026-02-26 03:52:10,623 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-26T03:52:10.704 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x9c59fe2 0x9c5a5cb 0x9d43118 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:10.704 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0xcb7e94 0x11253bf 0xf2e0d1 0x9c50c5c 0x9c50f59 0x9c511cc 0x9c521cc 0x9c5a24f 0x9c5a5cb 0x9d43118 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:10.704 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x9c5135e 0x9c51a44 0x9c59eeb 0x9c5a5cb 0x9d43118 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:10.852 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x316e0b2 0x9c50ec9 0x9c511cc 0x9c51482 0x9c51f31 0x9c5242f 0x9c52a5f 0x9c52bc9 0x9d32f39 0x9d45d3c 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:10.852 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb0494 /lib64/libasan.so.8+0xb1437 0xb69757 0x1090878 0xf2e0dc 0x9d33b41 0x9d33e86 0x9d34209 0x9d44360 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:10.852 INFO:teuthology.orchestra.run.trial160.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151e0 0x360962d 0x9c63157 0x9c6325f 0x9d23007 0x9d2319c 0x9d2352e 0x9d24b15 0x9d25e30 0x9d26250 0x9d42adb 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:12.983 INFO:teuthology.orchestra.run.trial160.stderr:----------------------------------------------------- 2026-02-26T03:52:12.983 INFO:teuthology.orchestra.run.trial160.stderr:Suppressions used: 2026-02-26T03:52:12.983 INFO:teuthology.orchestra.run.trial160.stderr: count bytes template 2026-02-26T03:52:12.983 INFO:teuthology.orchestra.run.trial160.stderr: 1 8 MallocExtension::Register 2026-02-26T03:52:12.983 INFO:teuthology.orchestra.run.trial160.stderr:----------------------------------------------------- 2026-02-26T03:52:12.983 INFO:teuthology.orchestra.run.trial160.stderr: 2026-02-26T03:52:13.015 DEBUG:teuthology.orchestra.run.trial160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-26T03:52:13.040 DEBUG:teuthology.orchestra.run.trial195:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-26T03:52:13.066 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-26T03:52:13.066 INFO:tasks.ceph:role: osd.2 2026-02-26T03:52:13.066 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial195.front.sepia.ceph.com 2026-02-26T03:52:13.067 DEBUG:teuthology.orchestra.run.trial195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-26T03:52:13.131 INFO:teuthology.orchestra.run.trial195.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-26T03:52:13.131 INFO:teuthology.orchestra.run.trial195.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-26T03:52:13.131 INFO:teuthology.orchestra.run.trial195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-26T03:52:13.131 INFO:teuthology.orchestra.run.trial195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-26T03:52:13.131 INFO:teuthology.orchestra.run.trial195.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-26T03:52:13.131 INFO:teuthology.orchestra.run.trial195.stdout: = sunit=0 swidth=0 blks 2026-02-26T03:52:13.131 INFO:teuthology.orchestra.run.trial195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-26T03:52:13.132 INFO:teuthology.orchestra.run.trial195.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-26T03:52:13.132 INFO:teuthology.orchestra.run.trial195.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-26T03:52:13.132 INFO:teuthology.orchestra.run.trial195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-26T03:52:13.704 INFO:teuthology.orchestra.run.trial195.stdout:Discarding blocks...Done. 2026-02-26T03:52:13.706 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial195.front.sepia.ceph.com -o noatime 2026-02-26T03:52:13.706 DEBUG:teuthology.orchestra.run.trial195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2026-02-26T03:52:13.774 DEBUG:teuthology.orchestra.run.trial195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-26T03:52:13.841 DEBUG:teuthology.orchestra.run.trial195:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-26T03:52:13.906 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-26T03:52:13.906 INFO:tasks.ceph:role: osd.3 2026-02-26T03:52:13.906 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial195.front.sepia.ceph.com 2026-02-26T03:52:13.907 DEBUG:teuthology.orchestra.run.trial195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-26T03:52:13.972 INFO:teuthology.orchestra.run.trial195.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-26T03:52:13.972 INFO:teuthology.orchestra.run.trial195.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-26T03:52:13.972 INFO:teuthology.orchestra.run.trial195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-26T03:52:13.972 INFO:teuthology.orchestra.run.trial195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-26T03:52:13.972 INFO:teuthology.orchestra.run.trial195.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-26T03:52:13.972 INFO:teuthology.orchestra.run.trial195.stdout: = sunit=0 swidth=0 blks 2026-02-26T03:52:13.973 INFO:teuthology.orchestra.run.trial195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-26T03:52:13.973 INFO:teuthology.orchestra.run.trial195.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-26T03:52:13.973 INFO:teuthology.orchestra.run.trial195.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-26T03:52:13.973 INFO:teuthology.orchestra.run.trial195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-26T03:52:14.530 INFO:teuthology.orchestra.run.trial195.stdout:Discarding blocks...Done. 2026-02-26T03:52:14.533 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial195.front.sepia.ceph.com -o noatime 2026-02-26T03:52:14.533 DEBUG:teuthology.orchestra.run.trial195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2026-02-26T03:52:14.605 DEBUG:teuthology.orchestra.run.trial195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-26T03:52:14.672 DEBUG:teuthology.orchestra.run.trial195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:52:14.822 INFO:teuthology.orchestra.run.trial195.stderr:INFO  osd - OSD::main: parsing early config 2026-02-26T03:52:14.878 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:14,876 seastar - Reactor backend: linux-aio 2026-02-26T03:52:14.895 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:14,893 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:14.895 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:14,893 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:14.897 INFO:teuthology.orchestra.run.trial195.stderr:==37073==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:52:14.952 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:14,950 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-26T03:52:14.952 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:14,950 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-26T03:52:14.952 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:14,950 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-26T03:52:14.952 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:14,950 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-26T03:52:14.952 INFO:teuthology.orchestra.run.trial195.stderr:DEBUG 2026-02-26 03:52:14,950 [shard 0:main] seastar - reactor::drain 2026-02-26T03:52:17.000 INFO:teuthology.orchestra.run.trial195.stderr:----------------------------------------------------- 2026-02-26T03:52:17.000 INFO:teuthology.orchestra.run.trial195.stderr:Suppressions used: 2026-02-26T03:52:17.000 INFO:teuthology.orchestra.run.trial195.stderr: count bytes template 2026-02-26T03:52:17.001 INFO:teuthology.orchestra.run.trial195.stderr: 1 8 MallocExtension::Register 2026-02-26T03:52:17.001 INFO:teuthology.orchestra.run.trial195.stderr:----------------------------------------------------- 2026-02-26T03:52:17.001 INFO:teuthology.orchestra.run.trial195.stderr: 2026-02-26T03:52:17.019 INFO:teuthology.orchestra.run.trial195.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-26T03:52:17.019 INFO:teuthology.orchestra.run.trial195.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-26T03:52:17.075 INFO:teuthology.orchestra.run.trial195.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-26T03:52:17.075 INFO:teuthology.orchestra.run.trial195.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-26T03:52:17.082 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:17,080 seastar - Reactor backend: linux-aio 2026-02-26T03:52:17.093 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:17,091 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:17.093 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:17,092 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:17.096 INFO:teuthology.orchestra.run.trial195.stderr:==37071==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:52:17.096 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:17,095 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-26T03:52:17.154 INFO:teuthology.orchestra.run.trial195.stderr:DEBUG 2026-02-26 03:52:17,152 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-26T03:52:17.232 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x165c5 0x9d33527 0x9d33e86 0x9d34209 0x9d44360 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:17.232 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe1d59b 0x9ba45bf 0x9d1b824 0x9d1b91d 0x7be9c60 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:17.232 INFO:teuthology.orchestra.run.trial195.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff825d0a9c 0xffffffff827e76f7 0xffffffff827e794a 0xffffffff827ca6fd 0xffffffff827d4bab 0xffffffff827d500b 0xffffffff824910d2 0xffffffff83150451 0xffffffff83200ba2 2026-02-26T03:52:17.233 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x10906fc 0xf2e0dc 0x9c554b6 0x9c55df5 0x9d41dc5 0x9d42ce5 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:17.376 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb0901 /lib64/libasan.so.8+0xb1437 0xb69757 0x1090878 0xf2e0dc 0x9c50c5c 0x9c50f59 0x9c511cc 0x9c51482 0x9c51f31 0x9c5242f 0x9c52a5f 0x9c52bc9 0x9d32f39 0x9d45d3c 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:17.376 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x1125487 0xf2e0d1 0x9c50a8f 0x9c50f59 0x9c511cc 0x9c51482 0x9c51f31 0x9c5242f 0x9c52a5f 0x9c52bc9 0x9d32f39 0x9d45d3c 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:17.376 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe1d59b 0x9d258d6 0x9d25a92 0x9d25bfb 0x9d25d26 0x9d26250 0x9d42adb 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:17.376 INFO:teuthology.orchestra.run.trial195.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff82ab68d3 0xffffffff82620021 0xffffffff826203b0 0xffffffff8249525e 0xffffffff827e7269 0xffffffff827ca7d5 0xffffffff827d4bab 0xffffffff827d500b 0xffffffff824910d2 0xffffffff83150451 0xffffffff83200ba2 2026-02-26T03:52:19.535 INFO:teuthology.orchestra.run.trial195.stderr:----------------------------------------------------- 2026-02-26T03:52:19.535 INFO:teuthology.orchestra.run.trial195.stderr:Suppressions used: 2026-02-26T03:52:19.535 INFO:teuthology.orchestra.run.trial195.stderr: count bytes template 2026-02-26T03:52:19.535 INFO:teuthology.orchestra.run.trial195.stderr: 1 8 MallocExtension::Register 2026-02-26T03:52:19.535 INFO:teuthology.orchestra.run.trial195.stderr:----------------------------------------------------- 2026-02-26T03:52:19.535 INFO:teuthology.orchestra.run.trial195.stderr: 2026-02-26T03:52:19.566 DEBUG:teuthology.orchestra.run.trial195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-26T03:52:19.591 DEBUG:teuthology.orchestra.run.trial195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:52:19.740 INFO:teuthology.orchestra.run.trial195.stderr:INFO  osd - OSD::main: parsing early config 2026-02-26T03:52:19.796 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:19,795 seastar - Reactor backend: linux-aio 2026-02-26T03:52:19.808 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:19,806 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:19.808 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:19,806 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:19.810 INFO:teuthology.orchestra.run.trial195.stderr:==37140==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:52:19.865 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:19,863 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-26T03:52:19.866 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:19,863 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-26T03:52:19.866 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:19,863 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-26T03:52:19.866 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:19,863 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-26T03:52:19.866 INFO:teuthology.orchestra.run.trial195.stderr:DEBUG 2026-02-26 03:52:19,864 [shard 0:main] seastar - reactor::drain 2026-02-26T03:52:21.921 INFO:teuthology.orchestra.run.trial195.stderr:----------------------------------------------------- 2026-02-26T03:52:21.921 INFO:teuthology.orchestra.run.trial195.stderr:Suppressions used: 2026-02-26T03:52:21.921 INFO:teuthology.orchestra.run.trial195.stderr: count bytes template 2026-02-26T03:52:21.922 INFO:teuthology.orchestra.run.trial195.stderr: 1 8 MallocExtension::Register 2026-02-26T03:52:21.922 INFO:teuthology.orchestra.run.trial195.stderr:----------------------------------------------------- 2026-02-26T03:52:21.922 INFO:teuthology.orchestra.run.trial195.stderr: 2026-02-26T03:52:21.940 INFO:teuthology.orchestra.run.trial195.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-26T03:52:21.940 INFO:teuthology.orchestra.run.trial195.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-26T03:52:21.983 INFO:teuthology.orchestra.run.trial195.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-26T03:52:21.983 INFO:teuthology.orchestra.run.trial195.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-26T03:52:21.990 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:21,988 seastar - Reactor backend: linux-aio 2026-02-26T03:52:21.999 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:21,997 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:21.999 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:21,997 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-26T03:52:22.003 INFO:teuthology.orchestra.run.trial195.stderr:==37138==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-26T03:52:22.003 INFO:teuthology.orchestra.run.trial195.stderr:INFO 2026-02-26 03:52:22,001 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-26T03:52:22.066 INFO:teuthology.orchestra.run.trial195.stderr:DEBUG 2026-02-26 03:52:22,064 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-26T03:52:22.145 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x9d04680 0x9d04837 0x9d17695 0x9d1a23d 0x9d1a44e 0x9d1a4fa 0x9d42619 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:22.145 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cbf3 /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x157c7 0x10908ef 0xf2e0dc 0x9c50a8f 0x9c50f59 0x9c511cc 0x9c52ef3 0x9d38e4d 0x9d390bc 0x9d4334a 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:22.145 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6f4 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9b2559b 0x9ba2d36 0x9ba2e4c 0x9ba2f28 0x9ba31a4 0x9ba325d 0x9ba374b 0x9ba3b4f 0x9ba3e0c 0x9ba4c54 0x9d1b824 0x9d1b91d 0x7be9c60 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6fa06ea 0x6fa44eb 0x6faa095 0x7419054 0x743ba5c 0x728f369 0x728f461 0x736153a 0x73615f3 0x73617aa 0x743badd 0x728f4de 0x728f563 0x72bb301 0x7423ee4 0x7423f69 0x7424047 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:22.292 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe1d59b 0xe1d8f2 0x73d67bb 0x73d6b43 0x7159a3e 0x7be9f75 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:22.292 INFO:teuthology.orchestra.run.trial195.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff83162f0b 0xffffffff8278bc24 0xffffffff8278bd1b 0xffffffff827ca7ef 0xffffffff827d4bab 0xffffffff827d500b 0xffffffff824910d2 0xffffffff83150451 0xffffffff83200ba2 2026-02-26T03:52:22.292 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xc530 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9b2559b 0x9d0c53e 0x9d0c66c 0x9d3ea77 0x9d3f21a 0x9d496e7 0x78947dd 0x7bea228 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:22.292 INFO:teuthology.orchestra.run.trial195.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9e28178 0x9e28461 0x9e88dff 0x9e895f2 0x9e89860 0x9e899a8 0x9e89e67 /lib64/libc.so.6+0x3fc2f 0x1125359 0xf2e0d1 0x9c50a8f 0x9c50f59 0x9c511cc 0x9c51482 0x9c51a44 0x9c59eeb 0x9c5a5cb 0x9d43118 0x9d477c3 0x9d07d91 0x9d06d91 0x7bea2f1 0x7bf3e60 0x7a431c9 0x7a807bd 0x711f05d 0x6f43702 0x6f45a6a 0x7483f52 0x72a690f 0x72a6a11 0x72a6b09 0x7360706 0x73607bf 0x7360972 0x7360aaa 0x7360b25 0x7360baa 0x7360d05 0x7423961 0x7483fdb 0x728dda4 0x72a6c86 0x73b49a4 0x743027a 0x74302f5 0x7430370 0x74f3b82 0x75ebd86 0x9ee2cc7 0x9e36f9c 0x9f10737 0x9f78f1a 0x9f939aa 0x9f94026 0x26ea5d3 0xa5a1f30 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-26T03:52:24.390 INFO:teuthology.orchestra.run.trial195.stderr:----------------------------------------------------- 2026-02-26T03:52:24.390 INFO:teuthology.orchestra.run.trial195.stderr:Suppressions used: 2026-02-26T03:52:24.390 INFO:teuthology.orchestra.run.trial195.stderr: count bytes template 2026-02-26T03:52:24.390 INFO:teuthology.orchestra.run.trial195.stderr: 1 8 MallocExtension::Register 2026-02-26T03:52:24.390 INFO:teuthology.orchestra.run.trial195.stderr:----------------------------------------------------- 2026-02-26T03:52:24.391 INFO:teuthology.orchestra.run.trial195.stderr: 2026-02-26T03:52:24.422 DEBUG:teuthology.orchestra.run.trial195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-26T03:52:24.448 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-26T03:52:24.449 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:52:24.449 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-26T03:52:24.475 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:52:24.475 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-26T03:52:24.538 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:52:24.538 DEBUG:teuthology.orchestra.run.trial195:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-26T03:52:24.564 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:52:24.565 DEBUG:teuthology.orchestra.run.trial195:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-26T03:52:24.628 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:52:24.628 DEBUG:teuthology.orchestra.run.trial195:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-26T03:52:24.693 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:52:24.693 DEBUG:teuthology.orchestra.run.trial160:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-26T03:52:24.711 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:52:24.711 DEBUG:teuthology.orchestra.run.trial195:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2026-02-26T03:52:24.750 INFO:tasks.ceph:Adding keys to all mons... 2026-02-26T03:52:24.750 DEBUG:teuthology.orchestra.run.trial160:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-26T03:52:24.773 INFO:teuthology.orchestra.run.trial160.stdout:[osd.0] 2026-02-26T03:52:24.773 INFO:teuthology.orchestra.run.trial160.stdout: key = AQBlw59pgLGJLBAAMIGRRdbU2lzvEOKQVSb+Dg== 2026-02-26T03:52:24.773 INFO:teuthology.orchestra.run.trial160.stdout:[osd.1] 2026-02-26T03:52:24.773 INFO:teuthology.orchestra.run.trial160.stdout: key = AQBqw59peilhJRAAr+UH+5iJ7e8PJRk+5ZFiEg== 2026-02-26T03:52:24.774 INFO:teuthology.orchestra.run.trial160.stdout:[mgr.x] 2026-02-26T03:52:24.774 INFO:teuthology.orchestra.run.trial160.stdout: key = AQBhw59psYrEHhAAQIwaDmd6jcIw5ZGgMRJZeA== 2026-02-26T03:52:24.774 INFO:teuthology.orchestra.run.trial160.stdout:[osd.2] 2026-02-26T03:52:24.774 INFO:teuthology.orchestra.run.trial160.stdout: key = AQBxw59pyCtWCRAA5Y1ZA3OOns2wBdoFIWTHGQ== 2026-02-26T03:52:24.774 INFO:teuthology.orchestra.run.trial160.stdout:[osd.3] 2026-02-26T03:52:24.774 INFO:teuthology.orchestra.run.trial160.stdout: key = AQB2w59pImwVBBAAedzn9/RfRS3fIlSIKokI1g== 2026-02-26T03:52:24.774 INFO:teuthology.orchestra.run.trial160.stdout:[client.0] 2026-02-26T03:52:24.774 INFO:teuthology.orchestra.run.trial160.stdout: key = AQBhw59pgvZeIRAAhgw9n3lLUOxh5uyyMfFAUw== 2026-02-26T03:52:24.774 INFO:teuthology.orchestra.run.trial160.stdout:[client.1] 2026-02-26T03:52:24.774 INFO:teuthology.orchestra.run.trial160.stdout: key = AQBhw59p24SVJBAAqAfl5kztoucLcHBWZUorDg== 2026-02-26T03:52:24.774 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-26T03:52:24.863 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-26T03:52:24.912 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-26T03:52:24.961 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-26T03:52:25.010 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-26T03:52:25.059 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-26T03:52:25.109 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-26T03:52:25.160 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-26T03:52:25.160 DEBUG:teuthology.orchestra.run.trial160:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-26T03:52:25.185 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-26T03:52:25.270 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:25.266+0000 7f1d01662c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.270 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:25.266+0000 7f1d01662c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.282 DEBUG:teuthology.orchestra.run.trial160:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-26T03:52:25.306 DEBUG:teuthology.orchestra.run.trial160:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-26T03:52:25.363 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-26T03:52:25.363 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-26T03:52:25.364 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-26T03:52:25.405 INFO:tasks.ceph.mon.a:Started 2026-02-26T03:52:25.405 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-26T03:52:25.406 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-26T03:52:25.406 DEBUG:teuthology.orchestra.run.trial195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-26T03:52:25.409 INFO:tasks.ceph.mgr.x:Started 2026-02-26T03:52:25.409 DEBUG:tasks.ceph:set 0 configs 2026-02-26T03:52:25.409 DEBUG:teuthology.orchestra.run.trial160:> sudo ceph --cluster ceph config dump 2026-02-26T03:52:25.467 INFO:tasks.ceph.mon.a.trial160.stderr:2026-02-26T03:52:25.463+0000 7f939ee2dc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.467 INFO:tasks.ceph.mon.a.trial160.stderr:2026-02-26T03:52:25.463+0000 7f939ee2dc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.467 INFO:tasks.ceph.mon.a.trial160.stderr:2026-02-26T03:52:25.463+0000 7f939ee2dc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.469 INFO:tasks.ceph.mgr.x.trial195.stderr:2026-02-26T03:52:25.467+0000 7fda8d634fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.482 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:25.476+0000 7f885261a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.487 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:25.483+0000 7f885261a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.493 INFO:tasks.ceph.mgr.x.trial195.stderr:2026-02-26T03:52:25.491+0000 7fda8d634fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.493 INFO:tasks.ceph.mgr.x.trial195.stderr:2026-02-26T03:52:25.491+0000 7fda8d634fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.521 INFO:teuthology.orchestra.run.trial160.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-26T03:52:25.529 INFO:tasks.ceph:Setting crush tunables to default 2026-02-26T03:52:25.529 DEBUG:teuthology.orchestra.run.trial160:> sudo ceph --cluster ceph osd crush tunables default 2026-02-26T03:52:25.605 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:25.597+0000 7f047685a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.607 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:25.603+0000 7f047685a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.643 INFO:teuthology.orchestra.run.trial160.stderr:adjusted tunables profile to default 2026-02-26T03:52:25.651 INFO:tasks.ceph:check_enable_crimson: True 2026-02-26T03:52:25.651 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-26T03:52:25.651 DEBUG:teuthology.orchestra.run.trial160:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-26T03:52:25.726 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:25.719+0000 7f9ad397e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.728 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:25.725+0000 7f9ad397e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:25.767 INFO:tasks.ceph.mon.a.trial160.stderr:2026-02-26T03:52:25.764+0000 7f93961f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-26T03:52:25.768 INFO:tasks.ceph.mon.a.trial160.stderr:Please be aware that this feature is experimental, untested, 2026-02-26T03:52:25.768 INFO:tasks.ceph.mon.a.trial160.stderr:unsupported, and may result in data corruption, data loss, 2026-02-26T03:52:25.768 INFO:tasks.ceph.mon.a.trial160.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-26T03:52:25.768 INFO:tasks.ceph.mon.a.trial160.stderr:feature with important data. 2026-02-26T03:52:25.768 INFO:tasks.ceph.mon.a.trial160.stderr: 2026-02-26T03:52:26.329 INFO:tasks.ceph.mgr.x.trial195.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-26T03:52:26.329 INFO:tasks.ceph.mgr.x.trial195.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-26T03:52:26.329 INFO:tasks.ceph.mgr.x.trial195.stderr: from numpy import show_config as show_numpy_config 2026-02-26T03:52:26.654 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-26T03:52:26.655 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:52:26.655 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-26T03:52:26.679 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:52:26.679 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-26T03:52:26.744 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:52:26.745 DEBUG:teuthology.orchestra.run.trial195:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-26T03:52:26.770 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-26T03:52:26.770 DEBUG:teuthology.orchestra.run.trial195:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-26T03:52:26.834 DEBUG:teuthology.orchestra.run.trial195:> sudo ceph --cluster ceph osd new c85aaf43-ae33-4edc-9a34-00d4016144bd 0 2026-02-26T03:52:26.947 INFO:teuthology.orchestra.run.trial195.stderr:2026-02-26T03:52:26.942+0000 7f1b0e65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:26.949 INFO:teuthology.orchestra.run.trial195.stderr:2026-02-26T03:52:26.947+0000 7f1b0e65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:26.988 INFO:teuthology.orchestra.run.trial195.stdout:0 2026-02-26T03:52:26.996 DEBUG:teuthology.orchestra.run.trial195:> sudo ceph --cluster ceph osd new a1ae5c32-727b-4d90-9134-90e95306fbc9 1 2026-02-26T03:52:27.071 INFO:teuthology.orchestra.run.trial195.stderr:2026-02-26T03:52:27.065+0000 7f26546cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:27.073 INFO:teuthology.orchestra.run.trial195.stderr:2026-02-26T03:52:27.071+0000 7f26546cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:27.111 INFO:teuthology.orchestra.run.trial195.stdout:1 2026-02-26T03:52:27.119 DEBUG:teuthology.orchestra.run.trial195:> sudo ceph --cluster ceph osd new 52bfd162-35bb-44f6-93e8-c27dc35f6793 2 2026-02-26T03:52:27.193 INFO:teuthology.orchestra.run.trial195.stderr:2026-02-26T03:52:27.187+0000 7f1cfbd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:27.195 INFO:teuthology.orchestra.run.trial195.stderr:2026-02-26T03:52:27.192+0000 7f1cfbd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:27.233 INFO:teuthology.orchestra.run.trial195.stdout:2 2026-02-26T03:52:27.241 DEBUG:teuthology.orchestra.run.trial195:> sudo ceph --cluster ceph osd new 926c24e6-d674-4103-98b7-edb375331d49 3 2026-02-26T03:52:27.317 INFO:teuthology.orchestra.run.trial195.stderr:2026-02-26T03:52:27.312+0000 7f79dc6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:27.320 INFO:teuthology.orchestra.run.trial195.stderr:2026-02-26T03:52:27.317+0000 7f79dc6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:27.357 INFO:teuthology.orchestra.run.trial195.stdout:3 2026-02-26T03:52:27.365 DEBUG:teuthology.orchestra.run.trial160:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-26T03:52:27.396 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-26T03:52:27.396 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-26T03:52:27.438 INFO:tasks.ceph.osd.0:Started 2026-02-26T03:52:27.438 DEBUG:teuthology.orchestra.run.trial160:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-26T03:52:27.466 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-26T03:52:27.466 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-26T03:52:27.509 INFO:tasks.ceph.osd.1:Started 2026-02-26T03:52:27.509 DEBUG:teuthology.orchestra.run.trial195:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-26T03:52:27.539 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-26T03:52:27.539 DEBUG:teuthology.orchestra.run.trial195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-26T03:52:27.581 INFO:tasks.ceph.osd.2:Started 2026-02-26T03:52:27.581 DEBUG:teuthology.orchestra.run.trial195:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2026-02-26T03:52:27.611 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-26T03:52:27.612 DEBUG:teuthology.orchestra.run.trial195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2026-02-26T03:52:27.653 INFO:tasks.ceph.osd.3:Started 2026-02-26T03:52:27.654 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-26T03:52:27.731 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:27.725+0000 7f633a85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:27.733 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:27.730+0000 7f633a85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:27.766 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:27.767 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":7,"fsid":"2350f431-1ca3-4096-a5f6-1c0f74b874c1","created":"2026-02-26T03:52:25.477123+0000","modified":"2026-02-26T03:52:27.355218+0000","last_up_change":"0.000000","last_in_change":"2026-02-26T03:52:27.355218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"c85aaf43-ae33-4edc-9a34-00d4016144bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a1ae5c32-727b-4d90-9134-90e95306fbc9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"52bfd162-35bb-44f6-93e8-c27dc35f6793","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"926c24e6-d674-4103-98b7-edb375331d49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:52:27.775 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-26T03:52:27.775 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-26T03:52:28.178 DEBUG:teuthology.orchestra.run.trial160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-26T03:52:28.247 INFO:teuthology.misc.health.trial160.stderr:2026-02-26T03:52:28.241+0000 7f56ecd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:28.249 INFO:teuthology.misc.health.trial160.stderr:2026-02-26T03:52:28.246+0000 7f56ecd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:28.282 INFO:teuthology.misc.health.trial160.stdout: 2026-02-26T03:52:28.283 INFO:teuthology.misc.health.trial160.stdout:{"epoch":7,"fsid":"2350f431-1ca3-4096-a5f6-1c0f74b874c1","created":"2026-02-26T03:52:25.477123+0000","modified":"2026-02-26T03:52:27.355218+0000","last_up_change":"0.000000","last_in_change":"2026-02-26T03:52:27.355218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"c85aaf43-ae33-4edc-9a34-00d4016144bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a1ae5c32-727b-4d90-9134-90e95306fbc9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"52bfd162-35bb-44f6-93e8-c27dc35f6793","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"926c24e6-d674-4103-98b7-edb375331d49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:52:28.289 DEBUG:teuthology.misc:0 of 4 OSDs are up 2026-02-26T03:52:28.692 INFO:tasks.ceph.mgr.x.trial195.stderr:2026-02-26T03:52:28.690+0000 7fd9a8103640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-26T03:52:34.692 DEBUG:teuthology.orchestra.run.trial160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-26T03:52:34.764 INFO:teuthology.misc.health.trial160.stderr:2026-02-26T03:52:34.758+0000 7f288ed7e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:34.766 INFO:teuthology.misc.health.trial160.stderr:2026-02-26T03:52:34.763+0000 7f288ed7e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:34.880 INFO:teuthology.misc.health.trial160.stdout: 2026-02-26T03:52:34.880 INFO:teuthology.misc.health.trial160.stdout:{"epoch":11,"fsid":"2350f431-1ca3-4096-a5f6-1c0f74b874c1","created":"2026-02-26T03:52:25.477123+0000","modified":"2026-02-26T03:52:34.702059+0000","last_up_change":"2026-02-26T03:52:34.702059+0000","last_in_change":"2026-02-26T03:52:27.355218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-26T03:52:33.707548+0000","flags":294913,"flags_names":"hashpspool,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c85aaf43-ae33-4edc-9a34-00d4016144bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6806","nonce":817701548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6805","nonce":817701548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6808","nonce":817701548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6807","nonce":817701548}]},"public_addr":"10.20.193.160:6806/817701548","cluster_addr":"10.20.193.160:6805/817701548","heartbeat_back_addr":"10.20.193.160:6808/817701548","heartbeat_front_addr":"10.20.193.160:6807/817701548","state":["exists","up"]},{"osd":1,"uuid":"a1ae5c32-727b-4d90-9134-90e95306fbc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6800","nonce":405025136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6801","nonce":405025136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6802","nonce":405025136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6803","nonce":405025136}]},"public_addr":"10.20.193.160:6800/405025136","cluster_addr":"10.20.193.160:6801/405025136","heartbeat_back_addr":"10.20.193.160:6802/405025136","heartbeat_front_addr":"10.20.193.160:6803/405025136","state":["exists","up"]},{"osd":2,"uuid":"52bfd162-35bb-44f6-93e8-c27dc35f6793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6803","nonce":1669321335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6804","nonce":1669321335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6807","nonce":1669321335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6806","nonce":1669321335}]},"public_addr":"10.20.193.195:6803/1669321335","cluster_addr":"10.20.193.195:6804/1669321335","heartbeat_back_addr":"10.20.193.195:6807/1669321335","heartbeat_front_addr":"10.20.193.195:6806/1669321335","state":["exists","up"]},{"osd":3,"uuid":"926c24e6-d674-4103-98b7-edb375331d49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6805","nonce":3236882601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6802","nonce":3236882601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6808","nonce":3236882601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6809","nonce":3236882601}]},"public_addr":"10.20.193.195:6805/3236882601","cluster_addr":"10.20.193.195:6802/3236882601","heartbeat_back_addr":"10.20.193.195:6808/3236882601","heartbeat_front_addr":"10.20.193.195:6809/3236882601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:52:34.889 DEBUG:teuthology.misc:4 of 4 OSDs are up 2026-02-26T03:52:34.889 INFO:tasks.ceph:Creating RBD pool 2026-02-26T03:52:34.889 DEBUG:teuthology.orchestra.run.trial160:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-26T03:52:34.965 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:34.957+0000 7fa523fad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:34.967 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:34.964+0000 7fa523fad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:35.711 INFO:teuthology.orchestra.run.trial160.stderr:pool 'rbd' created 2026-02-26T03:52:35.726 DEBUG:teuthology.orchestra.run.trial160:> rbd --cluster ceph pool init rbd 2026-02-26T03:52:35.768 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:35.764+0000 7fca684fb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:35.768 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:35.764+0000 7fca684fb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:35.771 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:35.768+0000 7fca684fb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:38.752 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-26T03:52:38.753 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-26T03:52:38.754 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-26T03:52:38.869 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:38.863+0000 7f6b5c25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:38.872 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:38.868+0000 7f6b5c25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:38.983 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:38.992 INFO:teuthology.orchestra.run.trial160.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-26T03:52:38.992 INFO:tasks.ceph_manager:config epoch is 1 2026-02-26T03:52:38.993 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-26T03:52:38.993 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-26T03:52:38.993 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-26T03:52:39.069 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.063+0000 7f209cc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.071 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.068+0000 7f209cc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.215 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:39.225 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6800","nonce":3479950313},{"type":"v1","addr":"10.20.193.195:6801","nonce":3479950313}]},"active_addr":"10.20.193.195:6801/3479950313","active_change":"2026-02-26T03:52:27.683895+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.195:0","nonce":3925561366}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.195:0","nonce":1605015800}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.195:0","nonce":2520739126}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.195:0","nonce":3364629577}]}]} 2026-02-26T03:52:39.277 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-26T03:52:39.277 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-26T03:52:39.277 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-26T03:52:39.351 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.346+0000 7f770cc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.354 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.350+0000 7f770cc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.471 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:39.472 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":15,"fsid":"2350f431-1ca3-4096-a5f6-1c0f74b874c1","created":"2026-02-26T03:52:25.477123+0000","modified":"2026-02-26T03:52:38.720483+0000","last_up_change":"2026-02-26T03:52:34.702059+0000","last_in_change":"2026-02-26T03:52:27.355218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-26T03:52:33.707548+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-26T03:52:35.081419+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c85aaf43-ae33-4edc-9a34-00d4016144bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6806","nonce":817701548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6805","nonce":817701548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6808","nonce":817701548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6807","nonce":817701548}]},"public_addr":"10.20.193.160:6806/817701548","cluster_addr":"10.20.193.160:6805/817701548","heartbeat_back_addr":"10.20.193.160:6808/817701548","heartbeat_front_addr":"10.20.193.160:6807/817701548","state":["exists","up"]},{"osd":1,"uuid":"a1ae5c32-727b-4d90-9134-90e95306fbc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6800","nonce":405025136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6801","nonce":405025136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6802","nonce":405025136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6803","nonce":405025136}]},"public_addr":"10.20.193.160:6800/405025136","cluster_addr":"10.20.193.160:6801/405025136","heartbeat_back_addr":"10.20.193.160:6802/405025136","heartbeat_front_addr":"10.20.193.160:6803/405025136","state":["exists","up"]},{"osd":2,"uuid":"52bfd162-35bb-44f6-93e8-c27dc35f6793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6803","nonce":1669321335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6804","nonce":1669321335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6807","nonce":1669321335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6806","nonce":1669321335}]},"public_addr":"10.20.193.195:6803/1669321335","cluster_addr":"10.20.193.195:6804/1669321335","heartbeat_back_addr":"10.20.193.195:6807/1669321335","heartbeat_front_addr":"10.20.193.195:6806/1669321335","state":["exists","up"]},{"osd":3,"uuid":"926c24e6-d674-4103-98b7-edb375331d49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6805","nonce":3236882601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6802","nonce":3236882601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6808","nonce":3236882601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6809","nonce":3236882601}]},"public_addr":"10.20.193.195:6805/3236882601","cluster_addr":"10.20.193.195:6802/3236882601","heartbeat_back_addr":"10.20.193.195:6808/3236882601","heartbeat_front_addr":"10.20.193.195:6809/3236882601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:52:39.481 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-26T03:52:39.481 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-26T03:52:39.558 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.552+0000 7f6de1d74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.560 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.557+0000 7f6de1d74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.674 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:39.675 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":15,"fsid":"2350f431-1ca3-4096-a5f6-1c0f74b874c1","created":"2026-02-26T03:52:25.477123+0000","modified":"2026-02-26T03:52:38.720483+0000","last_up_change":"2026-02-26T03:52:34.702059+0000","last_in_change":"2026-02-26T03:52:27.355218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-26T03:52:33.707548+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-26T03:52:35.081419+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c85aaf43-ae33-4edc-9a34-00d4016144bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6806","nonce":817701548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6805","nonce":817701548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6808","nonce":817701548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6807","nonce":817701548}]},"public_addr":"10.20.193.160:6806/817701548","cluster_addr":"10.20.193.160:6805/817701548","heartbeat_back_addr":"10.20.193.160:6808/817701548","heartbeat_front_addr":"10.20.193.160:6807/817701548","state":["exists","up"]},{"osd":1,"uuid":"a1ae5c32-727b-4d90-9134-90e95306fbc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6800","nonce":405025136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6801","nonce":405025136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6802","nonce":405025136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6803","nonce":405025136}]},"public_addr":"10.20.193.160:6800/405025136","cluster_addr":"10.20.193.160:6801/405025136","heartbeat_back_addr":"10.20.193.160:6802/405025136","heartbeat_front_addr":"10.20.193.160:6803/405025136","state":["exists","up"]},{"osd":2,"uuid":"52bfd162-35bb-44f6-93e8-c27dc35f6793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6803","nonce":1669321335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6804","nonce":1669321335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6807","nonce":1669321335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6806","nonce":1669321335}]},"public_addr":"10.20.193.195:6803/1669321335","cluster_addr":"10.20.193.195:6804/1669321335","heartbeat_back_addr":"10.20.193.195:6807/1669321335","heartbeat_front_addr":"10.20.193.195:6806/1669321335","state":["exists","up"]},{"osd":3,"uuid":"926c24e6-d674-4103-98b7-edb375331d49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6805","nonce":3236882601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6802","nonce":3236882601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6808","nonce":3236882601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6809","nonce":3236882601}]},"public_addr":"10.20.193.195:6805/3236882601","cluster_addr":"10.20.193.195:6802/3236882601","heartbeat_back_addr":"10.20.193.195:6808/3236882601","heartbeat_front_addr":"10.20.193.195:6809/3236882601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:52:39.684 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-26T03:52:39.684 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-26T03:52:39.684 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-26T03:52:39.685 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-26T03:52:39.769 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.763+0000 7f88b00c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.769 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.762+0000 7f8690c11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.769 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.762+0000 7f329010b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.769 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.763+0000 7effe885a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.772 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.769+0000 7f88b00c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.772 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.769+0000 7effe885a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.772 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.769+0000 7f8690c11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.773 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.769+0000 7f329010b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.790 INFO:teuthology.orchestra.run.trial160.stdout:42949672966 2026-02-26T03:52:39.791 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-26T03:52:39.792 INFO:teuthology.orchestra.run.trial160.stdout:47244640261 2026-02-26T03:52:39.793 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-26T03:52:39.794 INFO:teuthology.orchestra.run.trial160.stdout:42949672966 2026-02-26T03:52:39.794 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-26T03:52:39.795 INFO:teuthology.orchestra.run.trial160.stdout:47244640261 2026-02-26T03:52:39.796 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-26T03:52:39.913 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.906+0000 7f6151fe4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.914 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.906+0000 7f1a15e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.914 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.908+0000 7f9cf6fe4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.916 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.912+0000 7f6151fe4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.916 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.912+0000 7f1a15e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.917 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.913+0000 7f9cf6fe4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.955 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.949+0000 7ff3c1e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:39.957 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:39.954+0000 7ff3c1e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:40.033 INFO:teuthology.orchestra.run.trial160.stdout:47244640261 2026-02-26T03:52:40.034 INFO:teuthology.orchestra.run.trial160.stdout:42949672966 2026-02-26T03:52:40.039 INFO:teuthology.orchestra.run.trial160.stdout:42949672966 2026-02-26T03:52:40.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.0 2026-02-26T03:52:40.046 DEBUG:teuthology.parallel:result is None 2026-02-26T03:52:40.046 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.1 2026-02-26T03:52:40.046 DEBUG:teuthology.parallel:result is None 2026-02-26T03:52:40.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.2 2026-02-26T03:52:40.052 DEBUG:teuthology.parallel:result is None 2026-02-26T03:52:40.071 INFO:teuthology.orchestra.run.trial160.stdout:47244640261 2026-02-26T03:52:40.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.3 2026-02-26T03:52:40.083 DEBUG:teuthology.parallel:result is None 2026-02-26T03:52:40.083 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-26T03:52:40.083 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:52:40.199 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:40.193+0000 7f4c09fdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:40.201 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:40.197+0000 7f4c09fdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:40.312 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:40.312 INFO:teuthology.orchestra.run.trial160.stderr:dumped all 2026-02-26T03:52:40.321 INFO:teuthology.orchestra.run.trial160.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2026-02-26T03:52:39.783414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83888736,"kb_used_data":83888736,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125826400,"statfs":{"total":214748299264,"available":128846233600,"internally_reserved":0,"allocated":85902065664,"data_stored":85902065664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.734207+0000","last_change":"2026-02-26T03:52:38.736727+0000","last_active":"2026-02-26T03:52:38.734207+0000","last_peered":"2026-02-26T03:52:38.734207+0000","last_clean":"2026-02-26T03:52:38.734207+0000","last_became_active":"2026-02-26T03:52:36.749872+0000","last_became_peered":"2026-02-26T03:52:36.749872+0000","last_unstale":"2026-02-26T03:52:38.734207+0000","last_undegraded":"2026-02-26T03:52:38.734207+0000","last_fullsized":"2026-02-26T03:52:38.734207+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.735232+0000","last_change":"2026-02-26T03:52:38.737272+0000","last_active":"2026-02-26T03:52:38.735232+0000","last_peered":"2026-02-26T03:52:38.735232+0000","last_clean":"2026-02-26T03:52:38.735232+0000","last_became_active":"2026-02-26T03:52:36.756413+0000","last_became_peered":"2026-02-26T03:52:36.756413+0000","last_unstale":"2026-02-26T03:52:38.735232+0000","last_undegraded":"2026-02-26T03:52:38.735232+0000","last_fullsized":"2026-02-26T03:52:38.735232+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.733656+0000","last_change":"2026-02-26T03:52:38.736040+0000","last_active":"2026-02-26T03:52:38.733656+0000","last_peered":"2026-02-26T03:52:38.733656+0000","last_clean":"2026-02-26T03:52:38.733656+0000","last_became_active":"2026-02-26T03:52:36.747323+0000","last_became_peered":"2026-02-26T03:52:36.747323+0000","last_unstale":"2026-02-26T03:52:38.733656+0000","last_undegraded":"2026-02-26T03:52:38.733656+0000","last_fullsized":"2026-02-26T03:52:38.733656+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.732804+0000","last_change":"2026-02-26T03:52:38.735590+0000","last_active":"2026-02-26T03:52:38.732804+0000","last_peered":"2026-02-26T03:52:38.732804+0000","last_clean":"2026-02-26T03:52:38.732804+0000","last_became_active":"2026-02-26T03:52:36.744870+0000","last_became_peered":"2026-02-26T03:52:36.744870+0000","last_unstale":"2026-02-26T03:52:38.732804+0000","last_undegraded":"2026-02-26T03:52:38.732804+0000","last_fullsized":"2026-02-26T03:52:38.732804+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.735604+0000","last_change":"2026-02-26T03:52:38.735604+0000","last_active":"2026-02-26T03:52:38.735604+0000","last_peered":"2026-02-26T03:52:38.735604+0000","last_clean":"2026-02-26T03:52:38.735604+0000","last_became_active":"2026-02-26T03:52:36.745315+0000","last_became_peered":"2026-02-26T03:52:36.745315+0000","last_unstale":"2026-02-26T03:52:38.735604+0000","last_undegraded":"2026-02-26T03:52:38.735604+0000","last_fullsized":"2026-02-26T03:52:38.735604+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.732369+0000","last_change":"2026-02-26T03:52:38.735296+0000","last_active":"2026-02-26T03:52:38.732369+0000","last_peered":"2026-02-26T03:52:38.732369+0000","last_clean":"2026-02-26T03:52:38.732369+0000","last_became_active":"2026-02-26T03:52:36.744177+0000","last_became_peered":"2026-02-26T03:52:36.744177+0000","last_unstale":"2026-02-26T03:52:38.732369+0000","last_undegraded":"2026-02-26T03:52:38.732369+0000","last_fullsized":"2026-02-26T03:52:38.732369+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.733291+0000","last_change":"2026-02-26T03:52:38.735698+0000","last_active":"2026-02-26T03:52:38.733291+0000","last_peered":"2026-02-26T03:52:38.733291+0000","last_clean":"2026-02-26T03:52:38.733291+0000","last_became_active":"2026-02-26T03:52:36.748522+0000","last_became_peered":"2026-02-26T03:52:36.748522+0000","last_unstale":"2026-02-26T03:52:38.733291+0000","last_undegraded":"2026-02-26T03:52:38.733291+0000","last_fullsized":"2026-02-26T03:52:38.733291+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.734242+0000","last_change":"2026-02-26T03:52:38.737043+0000","last_active":"2026-02-26T03:52:38.734242+0000","last_peered":"2026-02-26T03:52:38.734242+0000","last_clean":"2026-02-26T03:52:38.734242+0000","last_became_active":"2026-02-26T03:52:36.748803+0000","last_became_peered":"2026-02-26T03:52:36.748803+0000","last_unstale":"2026-02-26T03:52:38.734242+0000","last_undegraded":"2026-02-26T03:52:38.734242+0000","last_fullsized":"2026-02-26T03:52:38.734242+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":84,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.732736+0000","last_change":"2026-02-26T03:52:35.750083+0000","last_active":"2026-02-26T03:52:38.732736+0000","last_peered":"2026-02-26T03:52:38.732736+0000","last_clean":"2026-02-26T03:52:38.732736+0000","last_became_active":"2026-02-26T03:52:35.738890+0000","last_became_peered":"2026-02-26T03:52:35.738890+0000","last_unstale":"2026-02-26T03:52:38.732736+0000","last_undegraded":"2026-02-26T03:52:38.732736+0000","last_fullsized":"2026-02-26T03:52:38.732736+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_clean_scrub_stamp":"2026-02-26T03:52:34.702059+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128846233600,"internally_reserved":0,"allocated":85902065664,"data_stored":85902065664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972528,"kb_used_data":20972528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456256,"statfs":{"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972564,"kb_used_data":20972564,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456220,"statfs":{"total":53687074816,"available":32211169280,"internally_reserved":0,"allocated":21475905536,"data_stored":21475905536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211169280,"internally_reserved":0,"allocated":21475905536,"data_stored":21475905536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:52:40.323 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:52:40.399 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:40.393+0000 7f4221174640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:40.402 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:40.398+0000 7f4221174640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:40.514 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:40.515 INFO:teuthology.orchestra.run.trial160.stderr:dumped all 2026-02-26T03:52:40.523 INFO:teuthology.orchestra.run.trial160.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2026-02-26T03:52:39.783414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83888736,"kb_used_data":83888736,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125826400,"statfs":{"total":214748299264,"available":128846233600,"internally_reserved":0,"allocated":85902065664,"data_stored":85902065664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.734207+0000","last_change":"2026-02-26T03:52:38.736727+0000","last_active":"2026-02-26T03:52:38.734207+0000","last_peered":"2026-02-26T03:52:38.734207+0000","last_clean":"2026-02-26T03:52:38.734207+0000","last_became_active":"2026-02-26T03:52:36.749872+0000","last_became_peered":"2026-02-26T03:52:36.749872+0000","last_unstale":"2026-02-26T03:52:38.734207+0000","last_undegraded":"2026-02-26T03:52:38.734207+0000","last_fullsized":"2026-02-26T03:52:38.734207+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.735232+0000","last_change":"2026-02-26T03:52:38.737272+0000","last_active":"2026-02-26T03:52:38.735232+0000","last_peered":"2026-02-26T03:52:38.735232+0000","last_clean":"2026-02-26T03:52:38.735232+0000","last_became_active":"2026-02-26T03:52:36.756413+0000","last_became_peered":"2026-02-26T03:52:36.756413+0000","last_unstale":"2026-02-26T03:52:38.735232+0000","last_undegraded":"2026-02-26T03:52:38.735232+0000","last_fullsized":"2026-02-26T03:52:38.735232+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.733656+0000","last_change":"2026-02-26T03:52:38.736040+0000","last_active":"2026-02-26T03:52:38.733656+0000","last_peered":"2026-02-26T03:52:38.733656+0000","last_clean":"2026-02-26T03:52:38.733656+0000","last_became_active":"2026-02-26T03:52:36.747323+0000","last_became_peered":"2026-02-26T03:52:36.747323+0000","last_unstale":"2026-02-26T03:52:38.733656+0000","last_undegraded":"2026-02-26T03:52:38.733656+0000","last_fullsized":"2026-02-26T03:52:38.733656+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.732804+0000","last_change":"2026-02-26T03:52:38.735590+0000","last_active":"2026-02-26T03:52:38.732804+0000","last_peered":"2026-02-26T03:52:38.732804+0000","last_clean":"2026-02-26T03:52:38.732804+0000","last_became_active":"2026-02-26T03:52:36.744870+0000","last_became_peered":"2026-02-26T03:52:36.744870+0000","last_unstale":"2026-02-26T03:52:38.732804+0000","last_undegraded":"2026-02-26T03:52:38.732804+0000","last_fullsized":"2026-02-26T03:52:38.732804+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.735604+0000","last_change":"2026-02-26T03:52:38.735604+0000","last_active":"2026-02-26T03:52:38.735604+0000","last_peered":"2026-02-26T03:52:38.735604+0000","last_clean":"2026-02-26T03:52:38.735604+0000","last_became_active":"2026-02-26T03:52:36.745315+0000","last_became_peered":"2026-02-26T03:52:36.745315+0000","last_unstale":"2026-02-26T03:52:38.735604+0000","last_undegraded":"2026-02-26T03:52:38.735604+0000","last_fullsized":"2026-02-26T03:52:38.735604+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.732369+0000","last_change":"2026-02-26T03:52:38.735296+0000","last_active":"2026-02-26T03:52:38.732369+0000","last_peered":"2026-02-26T03:52:38.732369+0000","last_clean":"2026-02-26T03:52:38.732369+0000","last_became_active":"2026-02-26T03:52:36.744177+0000","last_became_peered":"2026-02-26T03:52:36.744177+0000","last_unstale":"2026-02-26T03:52:38.732369+0000","last_undegraded":"2026-02-26T03:52:38.732369+0000","last_fullsized":"2026-02-26T03:52:38.732369+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.733291+0000","last_change":"2026-02-26T03:52:38.735698+0000","last_active":"2026-02-26T03:52:38.733291+0000","last_peered":"2026-02-26T03:52:38.733291+0000","last_clean":"2026-02-26T03:52:38.733291+0000","last_became_active":"2026-02-26T03:52:36.748522+0000","last_became_peered":"2026-02-26T03:52:36.748522+0000","last_unstale":"2026-02-26T03:52:38.733291+0000","last_undegraded":"2026-02-26T03:52:38.733291+0000","last_fullsized":"2026-02-26T03:52:38.733291+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.734242+0000","last_change":"2026-02-26T03:52:38.737043+0000","last_active":"2026-02-26T03:52:38.734242+0000","last_peered":"2026-02-26T03:52:38.734242+0000","last_clean":"2026-02-26T03:52:38.734242+0000","last_became_active":"2026-02-26T03:52:36.748803+0000","last_became_peered":"2026-02-26T03:52:36.748803+0000","last_unstale":"2026-02-26T03:52:38.734242+0000","last_undegraded":"2026-02-26T03:52:38.734242+0000","last_fullsized":"2026-02-26T03:52:38.734242+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":84,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-26T03:52:38.732736+0000","last_change":"2026-02-26T03:52:35.750083+0000","last_active":"2026-02-26T03:52:38.732736+0000","last_peered":"2026-02-26T03:52:38.732736+0000","last_clean":"2026-02-26T03:52:38.732736+0000","last_became_active":"2026-02-26T03:52:35.738890+0000","last_became_peered":"2026-02-26T03:52:35.738890+0000","last_unstale":"2026-02-26T03:52:38.732736+0000","last_undegraded":"2026-02-26T03:52:38.732736+0000","last_fullsized":"2026-02-26T03:52:38.732736+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_clean_scrub_stamp":"2026-02-26T03:52:34.702059+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128846233600,"internally_reserved":0,"allocated":85902065664,"data_stored":85902065664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972528,"kb_used_data":20972528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456256,"statfs":{"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971824,"kb_used_data":20971824,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456960,"statfs":{"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972564,"kb_used_data":20972564,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456220,"statfs":{"total":53687074816,"available":32211169280,"internally_reserved":0,"allocated":21475905536,"data_stored":21475905536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211169280,"internally_reserved":0,"allocated":21475905536,"data_stored":21475905536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211927040,"internally_reserved":0,"allocated":21475147776,"data_stored":21475147776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32211206144,"internally_reserved":0,"allocated":21475868672,"data_stored":21475868672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:52:40.524 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-26T03:52:40.524 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-26T03:52:40.524 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-26T03:52:40.525 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-26T03:52:40.601 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:40.595+0000 7f7e7050b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:40.603 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:52:40.599+0000 7f7e7050b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:52:40.741 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:52:40.741 INFO:teuthology.orchestra.run.trial160.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-26T03:52:40.751 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-26T03:52:40.752 INFO:teuthology.run_tasks:Running task workunit... 2026-02-26T03:52:40.771 INFO:tasks.workunit:Pulling workunits from ref 1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996 2026-02-26T03:52:40.772 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-26T03:52:40.772 DEBUG:teuthology.orchestra.run.trial160:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-26T03:52:40.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:52:40.788 INFO:teuthology.orchestra.run.trial160.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-26T03:52:40.788 DEBUG:teuthology.orchestra.run.trial160:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-26T03:52:40.844 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-26T03:52:40.845 DEBUG:teuthology.orchestra.run.trial160:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-26T03:52:40.901 INFO:tasks.workunit:timeout=1h 2026-02-26T03:52:40.901 INFO:tasks.workunit:cleanup=True 2026-02-26T03:52:40.902 DEBUG:teuthology.orchestra.run.trial160:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996 2026-02-26T03:52:40.965 INFO:tasks.workunit.client.0.trial160.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr:Note: switching to '1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996'. 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr: 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr:state without impacting any branches by switching back to a branch. 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr: 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr: 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr: git switch -c 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr: 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr:Or undo this operation with: 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr: 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr: git switch - 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr: 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr: 2026-02-26T03:53:03.266 INFO:tasks.workunit.client.0.trial160.stderr:HEAD is now at 1f87b69b1a8 Merge pull request #67522 from rhcs-dashboard/notification-banner-simple-mode 2026-02-26T03:53:03.267 DEBUG:teuthology.orchestra.run.trial160:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-26T03:53:03.324 INFO:tasks.workunit.client.0.trial160.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-26T03:53:03.327 INFO:tasks.workunit.client.0.trial160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-26T03:53:03.327 INFO:tasks.workunit.client.0.trial160.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-26T03:53:03.423 INFO:tasks.workunit.client.0.trial160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-26T03:53:03.455 INFO:tasks.workunit.client.0.trial160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-26T03:53:03.481 INFO:tasks.workunit.client.0.trial160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-26T03:53:03.482 INFO:tasks.workunit.client.0.trial160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-26T03:53:03.483 INFO:tasks.workunit.client.0.trial160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-26T03:53:03.507 INFO:tasks.workunit.client.0.trial160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-26T03:53:03.511 DEBUG:teuthology.orchestra.run.trial160:> set -ex 2026-02-26T03:53:03.511 DEBUG:teuthology.orchestra.run.trial160:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-26T03:53:03.569 INFO:tasks.workunit:Running workunits matching rados/test_python.sh on client.0... 2026-02-26T03:53:03.572 INFO:tasks.workunit:Running workunit rados/test_python.sh... 2026-02-26T03:53:03.572 DEBUG:teuthology.orchestra.run.trial160:workunit test rados/test_python.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f87b69b1a8041112e26ae0ea3c0a2a0fd14d996 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 1h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh -m 'not (wait or tier or ec)' 2026-02-26T03:53:03.628 INFO:tasks.workunit.client.0.trial160.stderr:+ ceph osd pool create rbd 2026-02-26T03:53:03.682 INFO:tasks.workunit.client.0.trial160.stderr:2026-02-26T03:53:03.675+0000 7fd566ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:53:03.684 INFO:tasks.workunit.client.0.trial160.stderr:2026-02-26T03:53:03.681+0000 7fd566ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:53:03.854 INFO:tasks.workunit.client.0.trial160.stderr:pool 'rbd' already exists 2026-02-26T03:53:03.862 INFO:tasks.workunit.client.0.trial160.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh 2026-02-26T03:53:03.863 INFO:tasks.workunit.client.0.trial160.stderr:+ python3 -m pytest -v /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/../../../src/test/pybind/test_rados.py -m 'not (wait or tier or ec)' 2026-02-26T03:53:03.939 INFO:tasks.workunit.client.0.trial160.stdout:============================= test session starts ============================== 2026-02-26T03:53:03.939 INFO:tasks.workunit.client.0.trial160.stdout:platform linux -- Python 3.9.25, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3 2026-02-26T03:53:03.939 INFO:tasks.workunit.client.0.trial160.stdout:cachedir: .pytest_cache 2026-02-26T03:53:03.939 INFO:tasks.workunit.client.0.trial160.stdout:rootdir: /home/ubuntu/cephtest/clone.client.0/src/test/pybind, configfile: pytest.ini 2026-02-26T03:53:04.078 INFO:tasks.workunit.client.0.trial160.stdout:collecting ... collected 91 items / 5 deselected / 86 selected 2026-02-26T03:53:04.078 INFO:tasks.workunit.client.0.trial160.stdout: 2026-02-26T03:53:04.084 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_rados_init_error PASSED [ 1%] 2026-02-26T03:53:04.134 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_rados_init PASSED [ 2%] 2026-02-26T03:53:04.146 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_ioctx_context_manager PASSED [ 3%] 2026-02-26T03:53:04.153 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_parse_argv PASSED [ 4%] 2026-02-26T03:53:04.156 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_parse_argv_empty_str PASSED [ 5%] 2026-02-26T03:53:04.163 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_configuring PASSED [ 6%] 2026-02-26T03:53:04.176 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_connected PASSED [ 8%] 2026-02-26T03:53:04.188 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_shutdown PASSED [ 9%] 2026-02-26T03:53:04.204 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_ping_monitor PASSED [ 10%] 2026-02-26T03:53:04.218 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_annotations PASSED [ 11%] 2026-02-26T03:53:05.815 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_create PASSED [ 12%] 2026-02-26T03:53:07.823 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_create_utf8 PASSED [ 13%] 2026-02-26T03:53:09.831 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_pool_lookup_utf8 PASSED [ 15%] 2026-02-26T03:53:11.839 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_eexist PASSED [ 16%] 2026-02-26T03:53:19.874 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_list_pools PASSED [ 17%] 2026-02-26T03:53:19.893 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_fsid PASSED [ 18%] 2026-02-26T03:53:20.879 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_blocklist_add PASSED [ 19%] 2026-02-26T03:53:20.895 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_cluster_stats PASSED [ 20%] 2026-02-26T03:53:20.910 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_monitor_log PASSED [ 22%] 2026-02-26T03:53:23.887 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_last_version PASSED [ 23%] 2026-02-26T03:53:26.489 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_stats PASSED [ 24%] 2026-02-26T03:53:30.498 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write PASSED [ 25%] 2026-02-26T03:53:34.516 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_full PASSED [ 26%] 2026-02-26T03:53:38.531 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_writesame PASSED [ 27%] 2026-02-26T03:53:42.549 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_append PASSED [ 29%] 2026-02-26T03:53:46.563 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_zeros PASSED [ 30%] 2026-02-26T03:53:50.574 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_trunc PASSED [ 31%] 2026-02-26T03:53:54.597 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_cmpext PASSED [ 32%] 2026-02-26T03:53:58.502 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_objects_empty PASSED [ 33%] 2026-02-26T03:54:02.518 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_read_crc PASSED [ 34%] 2026-02-26T03:54:06.528 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_objects PASSED [ 36%] 2026-02-26T03:54:10.498 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_ns_objects PASSED [ 37%] 2026-02-26T03:54:14.510 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_xattrs PASSED [ 38%] 2026-02-26T03:54:18.522 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_obj_xattrs PASSED [ 39%] 2026-02-26T03:54:21.502 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_pool_id PASSED [ 40%] 2026-02-26T03:54:24.510 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_pool_name PASSED [ 41%] 2026-02-26T03:54:29.528 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_create_snap PASSED [ 43%] 2026-02-26T03:54:32.507 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_snaps_empty PASSED [ 44%] 2026-02-26T03:54:38.527 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_snaps PASSED [ 45%] 2026-02-26T03:54:42.508 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_lookup_snap PASSED [ 46%] 2026-02-26T03:54:46.524 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_timestamp PASSED [ 47%] 2026-02-26T03:54:51.508 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_remove_snap PASSED [ 48%] 2026-02-26T03:54:57.528 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_rollback PASSED [ 50%] 2026-02-26T03:55:03.548 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_rollback_removed PASSED [ 51%] 2026-02-26T03:55:09.574 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_read PASSED [ 52%] 2026-02-26T03:55:13.518 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_set_omap PASSED [ 53%] 2026-02-26T03:55:17.531 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_set_omap_aio PASSED [ 54%] 2026-02-26T03:55:21.515 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_ops PASSED [ 55%] 2026-02-26T03:55:25.525 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_execute_op PASSED [ 56%] 2026-02-26T03:55:29.543 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_writesame_op PASSED [ 58%] 2026-02-26T03:55:33.519 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_omap_vals_by_keys PASSED [ 59%] 2026-02-26T03:55:37.536 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_omap_keys PASSED [ 60%] 2026-02-26T03:55:41.519 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_clear_omap PASSED [ 61%] 2026-02-26T03:55:45.535 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_remove_omap_range2 PASSED [ 62%] 2026-02-26T03:55:49.552 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_omap_cmp PASSED [ 63%] 2026-02-26T03:55:53.526 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_cmpext_op PASSED [ 65%] 2026-02-26T03:55:57.542 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_xattrs_op PASSED [ 66%] 2026-02-26T03:56:00.522 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_locator PASSED [ 67%] 2026-02-26T03:56:04.533 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_operate_aio_write_op PASSED [ 68%] 2026-02-26T03:56:08.551 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write PASSED [ 69%] 2026-02-26T03:56:12.568 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_cmpext PASSED [ 70%] 2026-02-26T03:56:16.583 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_rmxattr PASSED [ 72%] 2026-02-26T03:56:20.526 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write_no_comp_ref PASSED [ 73%] 2026-02-26T03:56:24.539 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_append PASSED [ 74%] 2026-02-26T03:56:28.556 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write_full PASSED [ 75%] 2026-02-26T03:56:32.577 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_writesame PASSED [ 76%] 2026-02-26T03:56:36.594 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_stat PASSED [ 77%] 2026-02-26T03:56:40.531 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_remove PASSED [ 79%] 2026-02-26T03:56:44.541 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_lock PASSED [ 80%] 2026-02-26T03:56:48.556 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_execute PASSED [ 81%] 2026-02-26T03:56:52.573 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_execute PASSED [ 82%] 2026-02-26T03:56:56.585 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_setxattr PASSED [ 83%] 2026-02-26T03:57:05.548 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_applications PASSED [ 84%] 2026-02-26T03:57:08.558 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_service_daemon PASSED [ 86%] 2026-02-26T03:57:11.539 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_alignment PASSED [ 87%] 2026-02-26T03:57:14.543 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx2::test_get_last_version PASSED [ 88%] 2026-02-26T03:57:17.557 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx2::test_get_stats PASSED [ 89%] 2026-02-26T03:57:20.540 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_read PASSED [ 90%] 2026-02-26T03:57:23.548 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_seek PASSED [ 91%] 2026-02-26T03:57:26.559 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_write PASSED [ 93%] 2026-02-26T03:57:34.582 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoCtxSelfManagedSnaps::test PASSED [ 94%] 2026-02-26T03:57:34.600 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_monmap_dump PASSED [ 95%] 2026-02-26T03:57:34.671 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_osd_bench PASSED [ 96%] 2026-02-26T03:57:35.591 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_ceph_osd_pool_create_utf8 PASSED [ 97%] 2026-02-26T03:57:41.617 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestWatchNotify::test PASSED [ 98%] 2026-02-26T03:57:47.633 INFO:tasks.workunit.client.0.trial160.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestWatchNotify::test_aio_notify PASSED [100%] 2026-02-26T03:57:47.634 INFO:tasks.workunit.client.0.trial160.stdout: 2026-02-26T03:57:47.634 INFO:tasks.workunit.client.0.trial160.stdout:================= 86 passed, 5 deselected in 283.69s (0:04:43) ================= 2026-02-26T03:57:47.648 INFO:tasks.workunit.client.0.trial160.stderr:+ exit 0 2026-02-26T03:57:47.649 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-26T03:57:47.649 DEBUG:teuthology.orchestra.run.trial160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-26T03:57:47.716 INFO:tasks.workunit:Stopping ["rados/test_python.sh -m 'not (wait or tier or ec)'"] on client.0... 2026-02-26T03:57:47.716 DEBUG:teuthology.orchestra.run.trial160:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-26T03:57:48.116 DEBUG:teuthology.parallel:result is None 2026-02-26T03:57:48.116 DEBUG:teuthology.orchestra.run.trial160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-26T03:57:48.142 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-26T03:57:48.143 DEBUG:teuthology.orchestra.run.trial160:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-26T03:57:48.198 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-26T03:57:48.199 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-26T03:57:48.209 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-26T03:57:48.209 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:57:48.315 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:48.308+0000 7f2b961f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:48.317 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:48.313+0000 7f2b961f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:48.433 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:57:48.434 INFO:teuthology.orchestra.run.trial160.stderr:dumped all 2026-02-26T03:57:48.442 INFO:teuthology.orchestra.run.trial160.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2026-02-26T03:57:47.740941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83898864,"kb_used_data":83898864,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125816272,"statfs":{"total":214748299264,"available":128835862528,"internally_reserved":0,"allocated":85912436736,"data_stored":85912436736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.128303"},"pg_stats":[{"pgid":"73.f","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.668015+0000","last_change":"2026-02-26T03:57:36.698466+0000","last_active":"2026-02-26T03:57:43.668015+0000","last_peered":"2026-02-26T03:57:43.668015+0000","last_clean":"2026-02-26T03:57:43.668015+0000","last_became_active":"2026-02-26T03:57:36.694282+0000","last_became_peered":"2026-02-26T03:57:36.694282+0000","last_unstale":"2026-02-26T03:57:43.668015+0000","last_undegraded":"2026-02-26T03:57:43.668015+0000","last_fullsized":"2026-02-26T03:57:43.668015+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.e","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.668337+0000","last_change":"2026-02-26T03:57:36.695055+0000","last_active":"2026-02-26T03:57:44.668337+0000","last_peered":"2026-02-26T03:57:44.668337+0000","last_clean":"2026-02-26T03:57:44.668337+0000","last_became_active":"2026-02-26T03:57:36.689826+0000","last_became_peered":"2026-02-26T03:57:36.689826+0000","last_unstale":"2026-02-26T03:57:44.668337+0000","last_undegraded":"2026-02-26T03:57:44.668337+0000","last_fullsized":"2026-02-26T03:57:44.668337+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.d","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666443+0000","last_change":"2026-02-26T03:57:36.688656+0000","last_active":"2026-02-26T03:57:43.666443+0000","last_peered":"2026-02-26T03:57:43.666443+0000","last_clean":"2026-02-26T03:57:43.666443+0000","last_became_active":"2026-02-26T03:57:36.672224+0000","last_became_peered":"2026-02-26T03:57:36.672224+0000","last_unstale":"2026-02-26T03:57:43.666443+0000","last_undegraded":"2026-02-26T03:57:43.666443+0000","last_fullsized":"2026-02-26T03:57:43.666443+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.c","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.667306+0000","last_change":"2026-02-26T03:57:36.669418+0000","last_active":"2026-02-26T03:57:43.667306+0000","last_peered":"2026-02-26T03:57:43.667306+0000","last_clean":"2026-02-26T03:57:43.667306+0000","last_became_active":"2026-02-26T03:57:36.665056+0000","last_became_peered":"2026-02-26T03:57:36.665056+0000","last_unstale":"2026-02-26T03:57:43.667306+0000","last_undegraded":"2026-02-26T03:57:43.667306+0000","last_fullsized":"2026-02-26T03:57:43.667306+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.b","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.667291+0000","last_change":"2026-02-26T03:57:36.700898+0000","last_active":"2026-02-26T03:57:43.667291+0000","last_peered":"2026-02-26T03:57:43.667291+0000","last_clean":"2026-02-26T03:57:43.667291+0000","last_became_active":"2026-02-26T03:57:36.696446+0000","last_became_peered":"2026-02-26T03:57:36.696446+0000","last_unstale":"2026-02-26T03:57:43.667291+0000","last_undegraded":"2026-02-26T03:57:43.667291+0000","last_fullsized":"2026-02-26T03:57:43.667291+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.a","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.668911+0000","last_change":"2026-02-26T03:57:36.722333+0000","last_active":"2026-02-26T03:57:44.668911+0000","last_peered":"2026-02-26T03:57:44.668911+0000","last_clean":"2026-02-26T03:57:44.668911+0000","last_became_active":"2026-02-26T03:57:36.712454+0000","last_became_peered":"2026-02-26T03:57:36.712454+0000","last_unstale":"2026-02-26T03:57:44.668911+0000","last_undegraded":"2026-02-26T03:57:44.668911+0000","last_fullsized":"2026-02-26T03:57:44.668911+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.9","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.672958+0000","last_change":"2026-02-26T03:57:36.688198+0000","last_active":"2026-02-26T03:57:44.672958+0000","last_peered":"2026-02-26T03:57:44.672958+0000","last_clean":"2026-02-26T03:57:44.672958+0000","last_became_active":"2026-02-26T03:57:36.681925+0000","last_became_peered":"2026-02-26T03:57:36.681925+0000","last_unstale":"2026-02-26T03:57:44.672958+0000","last_undegraded":"2026-02-26T03:57:44.672958+0000","last_fullsized":"2026-02-26T03:57:44.672958+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.8","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666310+0000","last_change":"2026-02-26T03:57:36.681425+0000","last_active":"2026-02-26T03:57:43.666310+0000","last_peered":"2026-02-26T03:57:43.666310+0000","last_clean":"2026-02-26T03:57:43.666310+0000","last_became_active":"2026-02-26T03:57:36.670570+0000","last_became_peered":"2026-02-26T03:57:36.670570+0000","last_unstale":"2026-02-26T03:57:43.666310+0000","last_undegraded":"2026-02-26T03:57:43.666310+0000","last_fullsized":"2026-02-26T03:57:43.666310+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.7","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.671429+0000","last_change":"2026-02-26T03:57:36.689553+0000","last_active":"2026-02-26T03:57:44.671429+0000","last_peered":"2026-02-26T03:57:44.671429+0000","last_clean":"2026-02-26T03:57:44.671429+0000","last_became_active":"2026-02-26T03:57:36.676014+0000","last_became_peered":"2026-02-26T03:57:36.676014+0000","last_unstale":"2026-02-26T03:57:44.671429+0000","last_undegraded":"2026-02-26T03:57:44.671429+0000","last_fullsized":"2026-02-26T03:57:44.671429+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.6","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666998+0000","last_change":"2026-02-26T03:57:36.718085+0000","last_active":"2026-02-26T03:57:43.666998+0000","last_peered":"2026-02-26T03:57:43.666998+0000","last_clean":"2026-02-26T03:57:43.666998+0000","last_became_active":"2026-02-26T03:57:36.702612+0000","last_became_peered":"2026-02-26T03:57:36.702612+0000","last_unstale":"2026-02-26T03:57:43.666998+0000","last_undegraded":"2026-02-26T03:57:43.666998+0000","last_fullsized":"2026-02-26T03:57:43.666998+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.5","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666079+0000","last_change":"2026-02-26T03:57:36.683624+0000","last_active":"2026-02-26T03:57:43.666079+0000","last_peered":"2026-02-26T03:57:43.666079+0000","last_clean":"2026-02-26T03:57:43.666079+0000","last_became_active":"2026-02-26T03:57:36.666470+0000","last_became_peered":"2026-02-26T03:57:36.666470+0000","last_unstale":"2026-02-26T03:57:43.666079+0000","last_undegraded":"2026-02-26T03:57:43.666079+0000","last_fullsized":"2026-02-26T03:57:43.666079+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.4","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666133+0000","last_change":"2026-02-26T03:57:36.707220+0000","last_active":"2026-02-26T03:57:43.666133+0000","last_peered":"2026-02-26T03:57:43.666133+0000","last_clean":"2026-02-26T03:57:43.666133+0000","last_became_active":"2026-02-26T03:57:36.691808+0000","last_became_peered":"2026-02-26T03:57:36.691808+0000","last_unstale":"2026-02-26T03:57:43.666133+0000","last_undegraded":"2026-02-26T03:57:43.666133+0000","last_fullsized":"2026-02-26T03:57:43.666133+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.3","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.673773+0000","last_change":"2026-02-26T03:57:36.680547+0000","last_active":"2026-02-26T03:57:44.673773+0000","last_peered":"2026-02-26T03:57:44.673773+0000","last_clean":"2026-02-26T03:57:44.673773+0000","last_became_active":"2026-02-26T03:57:36.671742+0000","last_became_peered":"2026-02-26T03:57:36.671742+0000","last_unstale":"2026-02-26T03:57:44.673773+0000","last_undegraded":"2026-02-26T03:57:44.673773+0000","last_fullsized":"2026-02-26T03:57:44.673773+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.2","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.668608+0000","last_change":"2026-02-26T03:57:36.688778+0000","last_active":"2026-02-26T03:57:44.668608+0000","last_peered":"2026-02-26T03:57:44.668608+0000","last_clean":"2026-02-26T03:57:44.668608+0000","last_became_active":"2026-02-26T03:57:36.680023+0000","last_became_peered":"2026-02-26T03:57:36.680023+0000","last_unstale":"2026-02-26T03:57:44.668608+0000","last_undegraded":"2026-02-26T03:57:44.668608+0000","last_fullsized":"2026-02-26T03:57:44.668608+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.1","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.673605+0000","last_change":"2026-02-26T03:57:36.698512+0000","last_active":"2026-02-26T03:57:44.673605+0000","last_peered":"2026-02-26T03:57:44.673605+0000","last_clean":"2026-02-26T03:57:44.673605+0000","last_became_active":"2026-02-26T03:57:36.684373+0000","last_became_peered":"2026-02-26T03:57:36.684373+0000","last_unstale":"2026-02-26T03:57:44.673605+0000","last_undegraded":"2026-02-26T03:57:44.673605+0000","last_fullsized":"2026-02-26T03:57:44.673605+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.0","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.668851+0000","last_change":"2026-02-26T03:57:36.699973+0000","last_active":"2026-02-26T03:57:43.668851+0000","last_peered":"2026-02-26T03:57:43.668851+0000","last_clean":"2026-02-26T03:57:43.668851+0000","last_became_active":"2026-02-26T03:57:36.687150+0000","last_became_peered":"2026-02-26T03:57:36.687150+0000","last_unstale":"2026-02-26T03:57:43.668851+0000","last_undegraded":"2026-02-26T03:57:43.668851+0000","last_fullsized":"2026-02-26T03:57:43.668851+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":921,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.671809+0000","last_change":"2026-02-26T03:52:35.750083+0000","last_active":"2026-02-26T03:57:44.671809+0000","last_peered":"2026-02-26T03:57:44.671809+0000","last_clean":"2026-02-26T03:57:44.671809+0000","last_became_active":"2026-02-26T03:52:35.738890+0000","last_became_peered":"2026-02-26T03:52:35.738890+0000","last_unstale":"2026-02-26T03:57:44.671809+0000","last_undegraded":"2026-02-26T03:57:44.671809+0000","last_fullsized":"2026-02-26T03:57:44.671809+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_clean_scrub_stamp":"2026-02-26T03:52:34.702059+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":853,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.667967+0000","last_change":"2026-02-26T03:52:38.737043+0000","last_active":"2026-02-26T03:57:44.667967+0000","last_peered":"2026-02-26T03:57:44.667967+0000","last_clean":"2026-02-26T03:57:44.667967+0000","last_became_active":"2026-02-26T03:52:36.748803+0000","last_became_peered":"2026-02-26T03:52:36.748803+0000","last_unstale":"2026-02-26T03:57:44.667967+0000","last_undegraded":"2026-02-26T03:57:44.667967+0000","last_fullsized":"2026-02-26T03:57:44.667967+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":849,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.669141+0000","last_change":"2026-02-26T03:52:38.735698+0000","last_active":"2026-02-26T03:57:43.669141+0000","last_peered":"2026-02-26T03:57:43.669141+0000","last_clean":"2026-02-26T03:57:43.669141+0000","last_became_active":"2026-02-26T03:52:36.748522+0000","last_became_peered":"2026-02-26T03:52:36.748522+0000","last_unstale":"2026-02-26T03:57:43.669141+0000","last_undegraded":"2026-02-26T03:57:43.669141+0000","last_fullsized":"2026-02-26T03:57:43.669141+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":852,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.667059+0000","last_change":"2026-02-26T03:52:38.735296+0000","last_active":"2026-02-26T03:57:44.667059+0000","last_peered":"2026-02-26T03:57:44.667059+0000","last_clean":"2026-02-26T03:57:44.667059+0000","last_became_active":"2026-02-26T03:52:36.744177+0000","last_became_peered":"2026-02-26T03:52:36.744177+0000","last_unstale":"2026-02-26T03:57:44.667059+0000","last_undegraded":"2026-02-26T03:57:44.667059+0000","last_fullsized":"2026-02-26T03:57:44.667059+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":851,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.669332+0000","last_change":"2026-02-26T03:52:38.735604+0000","last_active":"2026-02-26T03:57:43.669332+0000","last_peered":"2026-02-26T03:57:43.669332+0000","last_clean":"2026-02-26T03:57:43.669332+0000","last_became_active":"2026-02-26T03:52:36.745315+0000","last_became_peered":"2026-02-26T03:52:36.745315+0000","last_unstale":"2026-02-26T03:57:43.669332+0000","last_undegraded":"2026-02-26T03:57:43.669332+0000","last_fullsized":"2026-02-26T03:57:43.669332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":849,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666264+0000","last_change":"2026-02-26T03:52:38.735590+0000","last_active":"2026-02-26T03:57:43.666264+0000","last_peered":"2026-02-26T03:57:43.666264+0000","last_clean":"2026-02-26T03:57:43.666264+0000","last_became_active":"2026-02-26T03:52:36.744870+0000","last_became_peered":"2026-02-26T03:52:36.744870+0000","last_unstale":"2026-02-26T03:57:43.666264+0000","last_undegraded":"2026-02-26T03:57:43.666264+0000","last_fullsized":"2026-02-26T03:57:43.666264+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":852,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.672120+0000","last_change":"2026-02-26T03:52:38.736040+0000","last_active":"2026-02-26T03:57:44.672120+0000","last_peered":"2026-02-26T03:57:44.672120+0000","last_clean":"2026-02-26T03:57:44.672120+0000","last_became_active":"2026-02-26T03:52:36.747323+0000","last_became_peered":"2026-02-26T03:52:36.747323+0000","last_unstale":"2026-02-26T03:57:44.672120+0000","last_undegraded":"2026-02-26T03:57:44.672120+0000","last_fullsized":"2026-02-26T03:57:44.672120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":849,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.668681+0000","last_change":"2026-02-26T03:52:38.737272+0000","last_active":"2026-02-26T03:57:43.668681+0000","last_peered":"2026-02-26T03:57:43.668681+0000","last_clean":"2026-02-26T03:57:43.668681+0000","last_became_active":"2026-02-26T03:52:36.756413+0000","last_became_peered":"2026-02-26T03:52:36.756413+0000","last_unstale":"2026-02-26T03:57:43.668681+0000","last_undegraded":"2026-02-26T03:57:43.668681+0000","last_fullsized":"2026-02-26T03:57:43.668681+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":852,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.667067+0000","last_change":"2026-02-26T03:52:38.736727+0000","last_active":"2026-02-26T03:57:44.667067+0000","last_peered":"2026-02-26T03:57:44.667067+0000","last_clean":"2026-02-26T03:57:44.667067+0000","last_became_active":"2026-02-26T03:52:36.749872+0000","last_became_peered":"2026-02-26T03:52:36.749872+0000","last_unstale":"2026-02-26T03:57:44.667067+0000","last_undegraded":"2026-02-26T03:57:44.667067+0000","last_fullsized":"2026-02-26T03:57:44.667067+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":73,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128836132864,"internally_reserved":0,"allocated":85912166400,"data_stored":85912166400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128836165632,"internally_reserved":0,"allocated":85912133632,"data_stored":85912133632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20975024,"kb_used_data":20975024,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31453760,"statfs":{"total":53687074816,"available":32208650240,"internally_reserved":0,"allocated":21478424576,"data_stored":21478424576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974384,"kb_used_data":20974384,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454400,"statfs":{"total":53687074816,"available":32209305600,"internally_reserved":0,"allocated":21477769216,"data_stored":21477769216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974308,"kb_used_data":20974308,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454476,"statfs":{"total":53687074816,"available":32209383424,"internally_reserved":0,"allocated":21477691392,"data_stored":21477691392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20975148,"kb_used_data":20975148,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31453636,"statfs":{"total":53687074816,"available":32208523264,"internally_reserved":0,"allocated":21478551552,"data_stored":21478551552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32208547840,"internally_reserved":0,"allocated":21478526976,"data_stored":21478526976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32209494016,"internally_reserved":0,"allocated":21477580800,"data_stored":21477580800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32209350656,"internally_reserved":0,"allocated":21477724160,"data_stored":21477724160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":0,"total":53687074816,"available":32208531456,"internally_reserved":0,"allocated":21478543360,"data_stored":21478543360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":1,"total":53687074816,"available":32209494016,"internally_reserved":0,"allocated":21477580800,"data_stored":21477580800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":2,"total":53687074816,"available":32209334272,"internally_reserved":0,"allocated":21477740544,"data_stored":21477740544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":3,"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:57:48.444 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:57:48.520 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:48.514+0000 7f15f04cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:48.523 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:48.518+0000 7f15f04cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:48.634 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:57:48.635 INFO:teuthology.orchestra.run.trial160.stderr:dumped all 2026-02-26T03:57:48.644 INFO:teuthology.orchestra.run.trial160.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2026-02-26T03:57:47.740941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83898864,"kb_used_data":83898864,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125816272,"statfs":{"total":214748299264,"available":128835862528,"internally_reserved":0,"allocated":85912436736,"data_stored":85912436736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.128303"},"pg_stats":[{"pgid":"73.f","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.668015+0000","last_change":"2026-02-26T03:57:36.698466+0000","last_active":"2026-02-26T03:57:43.668015+0000","last_peered":"2026-02-26T03:57:43.668015+0000","last_clean":"2026-02-26T03:57:43.668015+0000","last_became_active":"2026-02-26T03:57:36.694282+0000","last_became_peered":"2026-02-26T03:57:36.694282+0000","last_unstale":"2026-02-26T03:57:43.668015+0000","last_undegraded":"2026-02-26T03:57:43.668015+0000","last_fullsized":"2026-02-26T03:57:43.668015+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.e","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.668337+0000","last_change":"2026-02-26T03:57:36.695055+0000","last_active":"2026-02-26T03:57:44.668337+0000","last_peered":"2026-02-26T03:57:44.668337+0000","last_clean":"2026-02-26T03:57:44.668337+0000","last_became_active":"2026-02-26T03:57:36.689826+0000","last_became_peered":"2026-02-26T03:57:36.689826+0000","last_unstale":"2026-02-26T03:57:44.668337+0000","last_undegraded":"2026-02-26T03:57:44.668337+0000","last_fullsized":"2026-02-26T03:57:44.668337+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.d","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666443+0000","last_change":"2026-02-26T03:57:36.688656+0000","last_active":"2026-02-26T03:57:43.666443+0000","last_peered":"2026-02-26T03:57:43.666443+0000","last_clean":"2026-02-26T03:57:43.666443+0000","last_became_active":"2026-02-26T03:57:36.672224+0000","last_became_peered":"2026-02-26T03:57:36.672224+0000","last_unstale":"2026-02-26T03:57:43.666443+0000","last_undegraded":"2026-02-26T03:57:43.666443+0000","last_fullsized":"2026-02-26T03:57:43.666443+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.c","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.667306+0000","last_change":"2026-02-26T03:57:36.669418+0000","last_active":"2026-02-26T03:57:43.667306+0000","last_peered":"2026-02-26T03:57:43.667306+0000","last_clean":"2026-02-26T03:57:43.667306+0000","last_became_active":"2026-02-26T03:57:36.665056+0000","last_became_peered":"2026-02-26T03:57:36.665056+0000","last_unstale":"2026-02-26T03:57:43.667306+0000","last_undegraded":"2026-02-26T03:57:43.667306+0000","last_fullsized":"2026-02-26T03:57:43.667306+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.b","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.667291+0000","last_change":"2026-02-26T03:57:36.700898+0000","last_active":"2026-02-26T03:57:43.667291+0000","last_peered":"2026-02-26T03:57:43.667291+0000","last_clean":"2026-02-26T03:57:43.667291+0000","last_became_active":"2026-02-26T03:57:36.696446+0000","last_became_peered":"2026-02-26T03:57:36.696446+0000","last_unstale":"2026-02-26T03:57:43.667291+0000","last_undegraded":"2026-02-26T03:57:43.667291+0000","last_fullsized":"2026-02-26T03:57:43.667291+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.a","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.668911+0000","last_change":"2026-02-26T03:57:36.722333+0000","last_active":"2026-02-26T03:57:44.668911+0000","last_peered":"2026-02-26T03:57:44.668911+0000","last_clean":"2026-02-26T03:57:44.668911+0000","last_became_active":"2026-02-26T03:57:36.712454+0000","last_became_peered":"2026-02-26T03:57:36.712454+0000","last_unstale":"2026-02-26T03:57:44.668911+0000","last_undegraded":"2026-02-26T03:57:44.668911+0000","last_fullsized":"2026-02-26T03:57:44.668911+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.9","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.672958+0000","last_change":"2026-02-26T03:57:36.688198+0000","last_active":"2026-02-26T03:57:44.672958+0000","last_peered":"2026-02-26T03:57:44.672958+0000","last_clean":"2026-02-26T03:57:44.672958+0000","last_became_active":"2026-02-26T03:57:36.681925+0000","last_became_peered":"2026-02-26T03:57:36.681925+0000","last_unstale":"2026-02-26T03:57:44.672958+0000","last_undegraded":"2026-02-26T03:57:44.672958+0000","last_fullsized":"2026-02-26T03:57:44.672958+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.8","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666310+0000","last_change":"2026-02-26T03:57:36.681425+0000","last_active":"2026-02-26T03:57:43.666310+0000","last_peered":"2026-02-26T03:57:43.666310+0000","last_clean":"2026-02-26T03:57:43.666310+0000","last_became_active":"2026-02-26T03:57:36.670570+0000","last_became_peered":"2026-02-26T03:57:36.670570+0000","last_unstale":"2026-02-26T03:57:43.666310+0000","last_undegraded":"2026-02-26T03:57:43.666310+0000","last_fullsized":"2026-02-26T03:57:43.666310+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.7","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.671429+0000","last_change":"2026-02-26T03:57:36.689553+0000","last_active":"2026-02-26T03:57:44.671429+0000","last_peered":"2026-02-26T03:57:44.671429+0000","last_clean":"2026-02-26T03:57:44.671429+0000","last_became_active":"2026-02-26T03:57:36.676014+0000","last_became_peered":"2026-02-26T03:57:36.676014+0000","last_unstale":"2026-02-26T03:57:44.671429+0000","last_undegraded":"2026-02-26T03:57:44.671429+0000","last_fullsized":"2026-02-26T03:57:44.671429+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.6","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666998+0000","last_change":"2026-02-26T03:57:36.718085+0000","last_active":"2026-02-26T03:57:43.666998+0000","last_peered":"2026-02-26T03:57:43.666998+0000","last_clean":"2026-02-26T03:57:43.666998+0000","last_became_active":"2026-02-26T03:57:36.702612+0000","last_became_peered":"2026-02-26T03:57:36.702612+0000","last_unstale":"2026-02-26T03:57:43.666998+0000","last_undegraded":"2026-02-26T03:57:43.666998+0000","last_fullsized":"2026-02-26T03:57:43.666998+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.5","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666079+0000","last_change":"2026-02-26T03:57:36.683624+0000","last_active":"2026-02-26T03:57:43.666079+0000","last_peered":"2026-02-26T03:57:43.666079+0000","last_clean":"2026-02-26T03:57:43.666079+0000","last_became_active":"2026-02-26T03:57:36.666470+0000","last_became_peered":"2026-02-26T03:57:36.666470+0000","last_unstale":"2026-02-26T03:57:43.666079+0000","last_undegraded":"2026-02-26T03:57:43.666079+0000","last_fullsized":"2026-02-26T03:57:43.666079+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.4","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666133+0000","last_change":"2026-02-26T03:57:36.707220+0000","last_active":"2026-02-26T03:57:43.666133+0000","last_peered":"2026-02-26T03:57:43.666133+0000","last_clean":"2026-02-26T03:57:43.666133+0000","last_became_active":"2026-02-26T03:57:36.691808+0000","last_became_peered":"2026-02-26T03:57:36.691808+0000","last_unstale":"2026-02-26T03:57:43.666133+0000","last_undegraded":"2026-02-26T03:57:43.666133+0000","last_fullsized":"2026-02-26T03:57:43.666133+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.3","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.673773+0000","last_change":"2026-02-26T03:57:36.680547+0000","last_active":"2026-02-26T03:57:44.673773+0000","last_peered":"2026-02-26T03:57:44.673773+0000","last_clean":"2026-02-26T03:57:44.673773+0000","last_became_active":"2026-02-26T03:57:36.671742+0000","last_became_peered":"2026-02-26T03:57:36.671742+0000","last_unstale":"2026-02-26T03:57:44.673773+0000","last_undegraded":"2026-02-26T03:57:44.673773+0000","last_fullsized":"2026-02-26T03:57:44.673773+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.2","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.668608+0000","last_change":"2026-02-26T03:57:36.688778+0000","last_active":"2026-02-26T03:57:44.668608+0000","last_peered":"2026-02-26T03:57:44.668608+0000","last_clean":"2026-02-26T03:57:44.668608+0000","last_became_active":"2026-02-26T03:57:36.680023+0000","last_became_peered":"2026-02-26T03:57:36.680023+0000","last_unstale":"2026-02-26T03:57:44.668608+0000","last_undegraded":"2026-02-26T03:57:44.668608+0000","last_fullsized":"2026-02-26T03:57:44.668608+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.1","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.673605+0000","last_change":"2026-02-26T03:57:36.698512+0000","last_active":"2026-02-26T03:57:44.673605+0000","last_peered":"2026-02-26T03:57:44.673605+0000","last_clean":"2026-02-26T03:57:44.673605+0000","last_became_active":"2026-02-26T03:57:36.684373+0000","last_became_peered":"2026-02-26T03:57:36.684373+0000","last_unstale":"2026-02-26T03:57:44.673605+0000","last_undegraded":"2026-02-26T03:57:44.673605+0000","last_fullsized":"2026-02-26T03:57:44.673605+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.0","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.668851+0000","last_change":"2026-02-26T03:57:36.699973+0000","last_active":"2026-02-26T03:57:43.668851+0000","last_peered":"2026-02-26T03:57:43.668851+0000","last_clean":"2026-02-26T03:57:43.668851+0000","last_became_active":"2026-02-26T03:57:36.687150+0000","last_became_peered":"2026-02-26T03:57:36.687150+0000","last_unstale":"2026-02-26T03:57:43.668851+0000","last_undegraded":"2026-02-26T03:57:43.668851+0000","last_fullsized":"2026-02-26T03:57:43.668851+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":921,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.671809+0000","last_change":"2026-02-26T03:52:35.750083+0000","last_active":"2026-02-26T03:57:44.671809+0000","last_peered":"2026-02-26T03:57:44.671809+0000","last_clean":"2026-02-26T03:57:44.671809+0000","last_became_active":"2026-02-26T03:52:35.738890+0000","last_became_peered":"2026-02-26T03:52:35.738890+0000","last_unstale":"2026-02-26T03:57:44.671809+0000","last_undegraded":"2026-02-26T03:57:44.671809+0000","last_fullsized":"2026-02-26T03:57:44.671809+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_clean_scrub_stamp":"2026-02-26T03:52:34.702059+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":853,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.667967+0000","last_change":"2026-02-26T03:52:38.737043+0000","last_active":"2026-02-26T03:57:44.667967+0000","last_peered":"2026-02-26T03:57:44.667967+0000","last_clean":"2026-02-26T03:57:44.667967+0000","last_became_active":"2026-02-26T03:52:36.748803+0000","last_became_peered":"2026-02-26T03:52:36.748803+0000","last_unstale":"2026-02-26T03:57:44.667967+0000","last_undegraded":"2026-02-26T03:57:44.667967+0000","last_fullsized":"2026-02-26T03:57:44.667967+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":849,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.669141+0000","last_change":"2026-02-26T03:52:38.735698+0000","last_active":"2026-02-26T03:57:43.669141+0000","last_peered":"2026-02-26T03:57:43.669141+0000","last_clean":"2026-02-26T03:57:43.669141+0000","last_became_active":"2026-02-26T03:52:36.748522+0000","last_became_peered":"2026-02-26T03:52:36.748522+0000","last_unstale":"2026-02-26T03:57:43.669141+0000","last_undegraded":"2026-02-26T03:57:43.669141+0000","last_fullsized":"2026-02-26T03:57:43.669141+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":852,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.667059+0000","last_change":"2026-02-26T03:52:38.735296+0000","last_active":"2026-02-26T03:57:44.667059+0000","last_peered":"2026-02-26T03:57:44.667059+0000","last_clean":"2026-02-26T03:57:44.667059+0000","last_became_active":"2026-02-26T03:52:36.744177+0000","last_became_peered":"2026-02-26T03:52:36.744177+0000","last_unstale":"2026-02-26T03:57:44.667059+0000","last_undegraded":"2026-02-26T03:57:44.667059+0000","last_fullsized":"2026-02-26T03:57:44.667059+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":851,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.669332+0000","last_change":"2026-02-26T03:52:38.735604+0000","last_active":"2026-02-26T03:57:43.669332+0000","last_peered":"2026-02-26T03:57:43.669332+0000","last_clean":"2026-02-26T03:57:43.669332+0000","last_became_active":"2026-02-26T03:52:36.745315+0000","last_became_peered":"2026-02-26T03:52:36.745315+0000","last_unstale":"2026-02-26T03:57:43.669332+0000","last_undegraded":"2026-02-26T03:57:43.669332+0000","last_fullsized":"2026-02-26T03:57:43.669332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":849,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666264+0000","last_change":"2026-02-26T03:52:38.735590+0000","last_active":"2026-02-26T03:57:43.666264+0000","last_peered":"2026-02-26T03:57:43.666264+0000","last_clean":"2026-02-26T03:57:43.666264+0000","last_became_active":"2026-02-26T03:52:36.744870+0000","last_became_peered":"2026-02-26T03:52:36.744870+0000","last_unstale":"2026-02-26T03:57:43.666264+0000","last_undegraded":"2026-02-26T03:57:43.666264+0000","last_fullsized":"2026-02-26T03:57:43.666264+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":852,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.672120+0000","last_change":"2026-02-26T03:52:38.736040+0000","last_active":"2026-02-26T03:57:44.672120+0000","last_peered":"2026-02-26T03:57:44.672120+0000","last_clean":"2026-02-26T03:57:44.672120+0000","last_became_active":"2026-02-26T03:52:36.747323+0000","last_became_peered":"2026-02-26T03:52:36.747323+0000","last_unstale":"2026-02-26T03:57:44.672120+0000","last_undegraded":"2026-02-26T03:57:44.672120+0000","last_fullsized":"2026-02-26T03:57:44.672120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":849,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.668681+0000","last_change":"2026-02-26T03:52:38.737272+0000","last_active":"2026-02-26T03:57:43.668681+0000","last_peered":"2026-02-26T03:57:43.668681+0000","last_clean":"2026-02-26T03:57:43.668681+0000","last_became_active":"2026-02-26T03:52:36.756413+0000","last_became_peered":"2026-02-26T03:52:36.756413+0000","last_unstale":"2026-02-26T03:57:43.668681+0000","last_undegraded":"2026-02-26T03:57:43.668681+0000","last_fullsized":"2026-02-26T03:57:43.668681+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":852,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.667067+0000","last_change":"2026-02-26T03:52:38.736727+0000","last_active":"2026-02-26T03:57:44.667067+0000","last_peered":"2026-02-26T03:57:44.667067+0000","last_clean":"2026-02-26T03:57:44.667067+0000","last_became_active":"2026-02-26T03:52:36.749872+0000","last_became_peered":"2026-02-26T03:52:36.749872+0000","last_unstale":"2026-02-26T03:57:44.667067+0000","last_undegraded":"2026-02-26T03:57:44.667067+0000","last_fullsized":"2026-02-26T03:57:44.667067+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":73,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128836132864,"internally_reserved":0,"allocated":85912166400,"data_stored":85912166400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128836165632,"internally_reserved":0,"allocated":85912133632,"data_stored":85912133632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20975024,"kb_used_data":20975024,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31453760,"statfs":{"total":53687074816,"available":32208650240,"internally_reserved":0,"allocated":21478424576,"data_stored":21478424576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974384,"kb_used_data":20974384,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454400,"statfs":{"total":53687074816,"available":32209305600,"internally_reserved":0,"allocated":21477769216,"data_stored":21477769216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974308,"kb_used_data":20974308,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454476,"statfs":{"total":53687074816,"available":32209383424,"internally_reserved":0,"allocated":21477691392,"data_stored":21477691392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20975148,"kb_used_data":20975148,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31453636,"statfs":{"total":53687074816,"available":32208523264,"internally_reserved":0,"allocated":21478551552,"data_stored":21478551552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32208547840,"internally_reserved":0,"allocated":21478526976,"data_stored":21478526976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32209494016,"internally_reserved":0,"allocated":21477580800,"data_stored":21477580800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32209350656,"internally_reserved":0,"allocated":21477724160,"data_stored":21477724160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":0,"total":53687074816,"available":32208531456,"internally_reserved":0,"allocated":21478543360,"data_stored":21478543360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":1,"total":53687074816,"available":32209494016,"internally_reserved":0,"allocated":21477580800,"data_stored":21477580800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":2,"total":53687074816,"available":32209334272,"internally_reserved":0,"allocated":21477740544,"data_stored":21477740544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":3,"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:57:48.646 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-26T03:57:48.647 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:57:48.725 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:48.719+0000 7f618fa1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:48.728 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:48.724+0000 7f618fa1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:48.840 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:57:48.840 INFO:teuthology.orchestra.run.trial160.stderr:dumped all 2026-02-26T03:57:48.851 INFO:teuthology.orchestra.run.trial160.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2026-02-26T03:57:47.740941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83898864,"kb_used_data":83898864,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125816272,"statfs":{"total":214748299264,"available":128835862528,"internally_reserved":0,"allocated":85912436736,"data_stored":85912436736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.128303"},"pg_stats":[{"pgid":"73.f","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.668015+0000","last_change":"2026-02-26T03:57:36.698466+0000","last_active":"2026-02-26T03:57:43.668015+0000","last_peered":"2026-02-26T03:57:43.668015+0000","last_clean":"2026-02-26T03:57:43.668015+0000","last_became_active":"2026-02-26T03:57:36.694282+0000","last_became_peered":"2026-02-26T03:57:36.694282+0000","last_unstale":"2026-02-26T03:57:43.668015+0000","last_undegraded":"2026-02-26T03:57:43.668015+0000","last_fullsized":"2026-02-26T03:57:43.668015+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.e","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.668337+0000","last_change":"2026-02-26T03:57:36.695055+0000","last_active":"2026-02-26T03:57:44.668337+0000","last_peered":"2026-02-26T03:57:44.668337+0000","last_clean":"2026-02-26T03:57:44.668337+0000","last_became_active":"2026-02-26T03:57:36.689826+0000","last_became_peered":"2026-02-26T03:57:36.689826+0000","last_unstale":"2026-02-26T03:57:44.668337+0000","last_undegraded":"2026-02-26T03:57:44.668337+0000","last_fullsized":"2026-02-26T03:57:44.668337+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.d","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666443+0000","last_change":"2026-02-26T03:57:36.688656+0000","last_active":"2026-02-26T03:57:43.666443+0000","last_peered":"2026-02-26T03:57:43.666443+0000","last_clean":"2026-02-26T03:57:43.666443+0000","last_became_active":"2026-02-26T03:57:36.672224+0000","last_became_peered":"2026-02-26T03:57:36.672224+0000","last_unstale":"2026-02-26T03:57:43.666443+0000","last_undegraded":"2026-02-26T03:57:43.666443+0000","last_fullsized":"2026-02-26T03:57:43.666443+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.c","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.667306+0000","last_change":"2026-02-26T03:57:36.669418+0000","last_active":"2026-02-26T03:57:43.667306+0000","last_peered":"2026-02-26T03:57:43.667306+0000","last_clean":"2026-02-26T03:57:43.667306+0000","last_became_active":"2026-02-26T03:57:36.665056+0000","last_became_peered":"2026-02-26T03:57:36.665056+0000","last_unstale":"2026-02-26T03:57:43.667306+0000","last_undegraded":"2026-02-26T03:57:43.667306+0000","last_fullsized":"2026-02-26T03:57:43.667306+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.b","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.667291+0000","last_change":"2026-02-26T03:57:36.700898+0000","last_active":"2026-02-26T03:57:43.667291+0000","last_peered":"2026-02-26T03:57:43.667291+0000","last_clean":"2026-02-26T03:57:43.667291+0000","last_became_active":"2026-02-26T03:57:36.696446+0000","last_became_peered":"2026-02-26T03:57:36.696446+0000","last_unstale":"2026-02-26T03:57:43.667291+0000","last_undegraded":"2026-02-26T03:57:43.667291+0000","last_fullsized":"2026-02-26T03:57:43.667291+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.a","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.668911+0000","last_change":"2026-02-26T03:57:36.722333+0000","last_active":"2026-02-26T03:57:44.668911+0000","last_peered":"2026-02-26T03:57:44.668911+0000","last_clean":"2026-02-26T03:57:44.668911+0000","last_became_active":"2026-02-26T03:57:36.712454+0000","last_became_peered":"2026-02-26T03:57:36.712454+0000","last_unstale":"2026-02-26T03:57:44.668911+0000","last_undegraded":"2026-02-26T03:57:44.668911+0000","last_fullsized":"2026-02-26T03:57:44.668911+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.9","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.672958+0000","last_change":"2026-02-26T03:57:36.688198+0000","last_active":"2026-02-26T03:57:44.672958+0000","last_peered":"2026-02-26T03:57:44.672958+0000","last_clean":"2026-02-26T03:57:44.672958+0000","last_became_active":"2026-02-26T03:57:36.681925+0000","last_became_peered":"2026-02-26T03:57:36.681925+0000","last_unstale":"2026-02-26T03:57:44.672958+0000","last_undegraded":"2026-02-26T03:57:44.672958+0000","last_fullsized":"2026-02-26T03:57:44.672958+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.8","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666310+0000","last_change":"2026-02-26T03:57:36.681425+0000","last_active":"2026-02-26T03:57:43.666310+0000","last_peered":"2026-02-26T03:57:43.666310+0000","last_clean":"2026-02-26T03:57:43.666310+0000","last_became_active":"2026-02-26T03:57:36.670570+0000","last_became_peered":"2026-02-26T03:57:36.670570+0000","last_unstale":"2026-02-26T03:57:43.666310+0000","last_undegraded":"2026-02-26T03:57:43.666310+0000","last_fullsized":"2026-02-26T03:57:43.666310+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.7","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.671429+0000","last_change":"2026-02-26T03:57:36.689553+0000","last_active":"2026-02-26T03:57:44.671429+0000","last_peered":"2026-02-26T03:57:44.671429+0000","last_clean":"2026-02-26T03:57:44.671429+0000","last_became_active":"2026-02-26T03:57:36.676014+0000","last_became_peered":"2026-02-26T03:57:36.676014+0000","last_unstale":"2026-02-26T03:57:44.671429+0000","last_undegraded":"2026-02-26T03:57:44.671429+0000","last_fullsized":"2026-02-26T03:57:44.671429+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.6","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666998+0000","last_change":"2026-02-26T03:57:36.718085+0000","last_active":"2026-02-26T03:57:43.666998+0000","last_peered":"2026-02-26T03:57:43.666998+0000","last_clean":"2026-02-26T03:57:43.666998+0000","last_became_active":"2026-02-26T03:57:36.702612+0000","last_became_peered":"2026-02-26T03:57:36.702612+0000","last_unstale":"2026-02-26T03:57:43.666998+0000","last_undegraded":"2026-02-26T03:57:43.666998+0000","last_fullsized":"2026-02-26T03:57:43.666998+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.5","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666079+0000","last_change":"2026-02-26T03:57:36.683624+0000","last_active":"2026-02-26T03:57:43.666079+0000","last_peered":"2026-02-26T03:57:43.666079+0000","last_clean":"2026-02-26T03:57:43.666079+0000","last_became_active":"2026-02-26T03:57:36.666470+0000","last_became_peered":"2026-02-26T03:57:36.666470+0000","last_unstale":"2026-02-26T03:57:43.666079+0000","last_undegraded":"2026-02-26T03:57:43.666079+0000","last_fullsized":"2026-02-26T03:57:43.666079+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.4","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666133+0000","last_change":"2026-02-26T03:57:36.707220+0000","last_active":"2026-02-26T03:57:43.666133+0000","last_peered":"2026-02-26T03:57:43.666133+0000","last_clean":"2026-02-26T03:57:43.666133+0000","last_became_active":"2026-02-26T03:57:36.691808+0000","last_became_peered":"2026-02-26T03:57:36.691808+0000","last_unstale":"2026-02-26T03:57:43.666133+0000","last_undegraded":"2026-02-26T03:57:43.666133+0000","last_fullsized":"2026-02-26T03:57:43.666133+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.3","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.673773+0000","last_change":"2026-02-26T03:57:36.680547+0000","last_active":"2026-02-26T03:57:44.673773+0000","last_peered":"2026-02-26T03:57:44.673773+0000","last_clean":"2026-02-26T03:57:44.673773+0000","last_became_active":"2026-02-26T03:57:36.671742+0000","last_became_peered":"2026-02-26T03:57:36.671742+0000","last_unstale":"2026-02-26T03:57:44.673773+0000","last_undegraded":"2026-02-26T03:57:44.673773+0000","last_fullsized":"2026-02-26T03:57:44.673773+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.2","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.668608+0000","last_change":"2026-02-26T03:57:36.688778+0000","last_active":"2026-02-26T03:57:44.668608+0000","last_peered":"2026-02-26T03:57:44.668608+0000","last_clean":"2026-02-26T03:57:44.668608+0000","last_became_active":"2026-02-26T03:57:36.680023+0000","last_became_peered":"2026-02-26T03:57:36.680023+0000","last_unstale":"2026-02-26T03:57:44.668608+0000","last_undegraded":"2026-02-26T03:57:44.668608+0000","last_fullsized":"2026-02-26T03:57:44.668608+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.1","version":"0'0","reported_seq":26,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.673605+0000","last_change":"2026-02-26T03:57:36.698512+0000","last_active":"2026-02-26T03:57:44.673605+0000","last_peered":"2026-02-26T03:57:44.673605+0000","last_clean":"2026-02-26T03:57:44.673605+0000","last_became_active":"2026-02-26T03:57:36.684373+0000","last_became_peered":"2026-02-26T03:57:36.684373+0000","last_unstale":"2026-02-26T03:57:44.673605+0000","last_undegraded":"2026-02-26T03:57:44.673605+0000","last_fullsized":"2026-02-26T03:57:44.673605+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.0","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.668851+0000","last_change":"2026-02-26T03:57:36.699973+0000","last_active":"2026-02-26T03:57:43.668851+0000","last_peered":"2026-02-26T03:57:43.668851+0000","last_clean":"2026-02-26T03:57:43.668851+0000","last_became_active":"2026-02-26T03:57:36.687150+0000","last_became_peered":"2026-02-26T03:57:36.687150+0000","last_unstale":"2026-02-26T03:57:43.668851+0000","last_undegraded":"2026-02-26T03:57:43.668851+0000","last_fullsized":"2026-02-26T03:57:43.668851+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":921,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.671809+0000","last_change":"2026-02-26T03:52:35.750083+0000","last_active":"2026-02-26T03:57:44.671809+0000","last_peered":"2026-02-26T03:57:44.671809+0000","last_clean":"2026-02-26T03:57:44.671809+0000","last_became_active":"2026-02-26T03:52:35.738890+0000","last_became_peered":"2026-02-26T03:52:35.738890+0000","last_unstale":"2026-02-26T03:57:44.671809+0000","last_undegraded":"2026-02-26T03:57:44.671809+0000","last_fullsized":"2026-02-26T03:57:44.671809+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_clean_scrub_stamp":"2026-02-26T03:52:34.702059+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":853,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.667967+0000","last_change":"2026-02-26T03:52:38.737043+0000","last_active":"2026-02-26T03:57:44.667967+0000","last_peered":"2026-02-26T03:57:44.667967+0000","last_clean":"2026-02-26T03:57:44.667967+0000","last_became_active":"2026-02-26T03:52:36.748803+0000","last_became_peered":"2026-02-26T03:52:36.748803+0000","last_unstale":"2026-02-26T03:57:44.667967+0000","last_undegraded":"2026-02-26T03:57:44.667967+0000","last_fullsized":"2026-02-26T03:57:44.667967+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":849,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.669141+0000","last_change":"2026-02-26T03:52:38.735698+0000","last_active":"2026-02-26T03:57:43.669141+0000","last_peered":"2026-02-26T03:57:43.669141+0000","last_clean":"2026-02-26T03:57:43.669141+0000","last_became_active":"2026-02-26T03:52:36.748522+0000","last_became_peered":"2026-02-26T03:52:36.748522+0000","last_unstale":"2026-02-26T03:57:43.669141+0000","last_undegraded":"2026-02-26T03:57:43.669141+0000","last_fullsized":"2026-02-26T03:57:43.669141+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":852,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.667059+0000","last_change":"2026-02-26T03:52:38.735296+0000","last_active":"2026-02-26T03:57:44.667059+0000","last_peered":"2026-02-26T03:57:44.667059+0000","last_clean":"2026-02-26T03:57:44.667059+0000","last_became_active":"2026-02-26T03:52:36.744177+0000","last_became_peered":"2026-02-26T03:52:36.744177+0000","last_unstale":"2026-02-26T03:57:44.667059+0000","last_undegraded":"2026-02-26T03:57:44.667059+0000","last_fullsized":"2026-02-26T03:57:44.667059+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":851,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.669332+0000","last_change":"2026-02-26T03:52:38.735604+0000","last_active":"2026-02-26T03:57:43.669332+0000","last_peered":"2026-02-26T03:57:43.669332+0000","last_clean":"2026-02-26T03:57:43.669332+0000","last_became_active":"2026-02-26T03:52:36.745315+0000","last_became_peered":"2026-02-26T03:52:36.745315+0000","last_unstale":"2026-02-26T03:57:43.669332+0000","last_undegraded":"2026-02-26T03:57:43.669332+0000","last_fullsized":"2026-02-26T03:57:43.669332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":849,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.666264+0000","last_change":"2026-02-26T03:52:38.735590+0000","last_active":"2026-02-26T03:57:43.666264+0000","last_peered":"2026-02-26T03:57:43.666264+0000","last_clean":"2026-02-26T03:57:43.666264+0000","last_became_active":"2026-02-26T03:52:36.744870+0000","last_became_peered":"2026-02-26T03:52:36.744870+0000","last_unstale":"2026-02-26T03:57:43.666264+0000","last_undegraded":"2026-02-26T03:57:43.666264+0000","last_fullsized":"2026-02-26T03:57:43.666264+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":852,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.672120+0000","last_change":"2026-02-26T03:52:38.736040+0000","last_active":"2026-02-26T03:57:44.672120+0000","last_peered":"2026-02-26T03:57:44.672120+0000","last_clean":"2026-02-26T03:57:44.672120+0000","last_became_active":"2026-02-26T03:52:36.747323+0000","last_became_peered":"2026-02-26T03:52:36.747323+0000","last_unstale":"2026-02-26T03:57:44.672120+0000","last_undegraded":"2026-02-26T03:57:44.672120+0000","last_fullsized":"2026-02-26T03:57:44.672120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":849,"reported_epoch":293,"state":"active+clean","last_fresh":"2026-02-26T03:57:43.668681+0000","last_change":"2026-02-26T03:52:38.737272+0000","last_active":"2026-02-26T03:57:43.668681+0000","last_peered":"2026-02-26T03:57:43.668681+0000","last_clean":"2026-02-26T03:57:43.668681+0000","last_became_active":"2026-02-26T03:52:36.756413+0000","last_became_peered":"2026-02-26T03:52:36.756413+0000","last_unstale":"2026-02-26T03:57:43.668681+0000","last_undegraded":"2026-02-26T03:57:43.668681+0000","last_fullsized":"2026-02-26T03:57:43.668681+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":852,"reported_epoch":294,"state":"active+clean","last_fresh":"2026-02-26T03:57:44.667067+0000","last_change":"2026-02-26T03:52:38.736727+0000","last_active":"2026-02-26T03:57:44.667067+0000","last_peered":"2026-02-26T03:57:44.667067+0000","last_clean":"2026-02-26T03:57:44.667067+0000","last_became_active":"2026-02-26T03:52:36.749872+0000","last_became_peered":"2026-02-26T03:52:36.749872+0000","last_unstale":"2026-02-26T03:57:44.667067+0000","last_undegraded":"2026-02-26T03:57:44.667067+0000","last_fullsized":"2026-02-26T03:57:44.667067+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":73,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128836132864,"internally_reserved":0,"allocated":85912166400,"data_stored":85912166400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128836165632,"internally_reserved":0,"allocated":85912133632,"data_stored":85912133632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20975024,"kb_used_data":20975024,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31453760,"statfs":{"total":53687074816,"available":32208650240,"internally_reserved":0,"allocated":21478424576,"data_stored":21478424576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974384,"kb_used_data":20974384,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454400,"statfs":{"total":53687074816,"available":32209305600,"internally_reserved":0,"allocated":21477769216,"data_stored":21477769216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974308,"kb_used_data":20974308,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454476,"statfs":{"total":53687074816,"available":32209383424,"internally_reserved":0,"allocated":21477691392,"data_stored":21477691392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20975148,"kb_used_data":20975148,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31453636,"statfs":{"total":53687074816,"available":32208523264,"internally_reserved":0,"allocated":21478551552,"data_stored":21478551552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32208547840,"internally_reserved":0,"allocated":21478526976,"data_stored":21478526976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32209494016,"internally_reserved":0,"allocated":21477580800,"data_stored":21477580800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32209350656,"internally_reserved":0,"allocated":21477724160,"data_stored":21477724160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":0,"total":53687074816,"available":32208531456,"internally_reserved":0,"allocated":21478543360,"data_stored":21478543360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":1,"total":53687074816,"available":32209494016,"internally_reserved":0,"allocated":21477580800,"data_stored":21477580800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":2,"total":53687074816,"available":32209334272,"internally_reserved":0,"allocated":21477740544,"data_stored":21477740544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":3,"total":53687074816,"available":32208773120,"internally_reserved":0,"allocated":21478301696,"data_stored":21478301696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:57:48.851 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-26T03:57:48.927 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:48.921+0000 7fd02af0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:48.930 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:48.926+0000 7fd02af0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:49.042 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:57:49.042 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":295,"fsid":"2350f431-1ca3-4096-a5f6-1c0f74b874c1","created":"2026-02-26T03:52:25.477123+0000","modified":"2026-02-26T03:57:47.624274+0000","last_up_change":"2026-02-26T03:52:34.702059+0000","last_in_change":"2026-02-26T03:52:27.355218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":75,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-26T03:52:33.707548+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-26T03:52:35.081419+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":73,"pool_name":"黅","create_time":"2026-02-26T03:57:34.685585+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"289","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c85aaf43-ae33-4edc-9a34-00d4016144bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":292,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6806","nonce":817701548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6805","nonce":817701548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6808","nonce":817701548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6807","nonce":817701548}]},"public_addr":"10.20.193.160:6806/817701548","cluster_addr":"10.20.193.160:6805/817701548","heartbeat_back_addr":"10.20.193.160:6808/817701548","heartbeat_front_addr":"10.20.193.160:6807/817701548","state":["exists","up"]},{"osd":1,"uuid":"a1ae5c32-727b-4d90-9134-90e95306fbc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":292,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6800","nonce":405025136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6801","nonce":405025136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6802","nonce":405025136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6803","nonce":405025136}]},"public_addr":"10.20.193.160:6800/405025136","cluster_addr":"10.20.193.160:6801/405025136","heartbeat_back_addr":"10.20.193.160:6802/405025136","heartbeat_front_addr":"10.20.193.160:6803/405025136","state":["exists","up"]},{"osd":2,"uuid":"52bfd162-35bb-44f6-93e8-c27dc35f6793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":292,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6803","nonce":1669321335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6804","nonce":1669321335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6807","nonce":1669321335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6806","nonce":1669321335}]},"public_addr":"10.20.193.195:6803/1669321335","cluster_addr":"10.20.193.195:6804/1669321335","heartbeat_back_addr":"10.20.193.195:6807/1669321335","heartbeat_front_addr":"10.20.193.195:6806/1669321335","state":["exists","up"]},{"osd":3,"uuid":"926c24e6-d674-4103-98b7-edb375331d49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":292,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6805","nonce":3236882601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6802","nonce":3236882601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6808","nonce":3236882601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.195:6809","nonce":3236882601}]},"public_addr":"10.20.193.195:6805/3236882601","cluster_addr":"10.20.193.195:6802/3236882601","heartbeat_back_addr":"10.20.193.195:6808/3236882601","heartbeat_front_addr":"10.20.193.195:6809/3236882601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]},{"pool":28,"snaps":[{"begin":1,"length":1}]},{"pool":33,"snaps":[{"begin":1,"length":1}]},{"pool":34,"snaps":[{"begin":1,"length":1}]},{"pool":35,"snaps":[{"begin":1,"length":1}]},{"pool":36,"snaps":[{"begin":1,"length":1}]},{"pool":72,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-26T03:57:50.052 INFO:tasks.ceph:Scrubbing osd.0 2026-02-26T03:57:50.052 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-26T03:57:50.133 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.124+0000 7f7b307c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.135 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.131+0000 7f7b307c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.145 INFO:teuthology.orchestra.run.trial160.stdout:{ 2026-02-26T03:57:50.145 INFO:teuthology.orchestra.run.trial160.stdout: "success": "" 2026-02-26T03:57:50.145 INFO:teuthology.orchestra.run.trial160.stdout:} 2026-02-26T03:57:50.153 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-26T03:57:50.231 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.224+0000 7f491d583640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.233 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.229+0000 7f491d583640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.344 INFO:teuthology.orchestra.run.trial160.stderr:instructed osd(s) 0 to deep-scrub 2026-02-26T03:57:50.358 INFO:tasks.ceph:Scrubbing osd.1 2026-02-26T03:57:50.358 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-26T03:57:50.436 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.427+0000 7fd274f93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.438 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.434+0000 7fd274f93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.448 INFO:teuthology.orchestra.run.trial160.stdout:{ 2026-02-26T03:57:50.448 INFO:teuthology.orchestra.run.trial160.stdout: "success": "" 2026-02-26T03:57:50.449 INFO:teuthology.orchestra.run.trial160.stdout:} 2026-02-26T03:57:50.456 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-26T03:57:50.535 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.528+0000 7f5a18846640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.537 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.533+0000 7f5a18846640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.649 INFO:teuthology.orchestra.run.trial160.stderr:instructed osd(s) 1 to deep-scrub 2026-02-26T03:57:50.662 INFO:tasks.ceph:Scrubbing osd.2 2026-02-26T03:57:50.662 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-26T03:57:50.742 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.733+0000 7ff2cc7cc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.744 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.740+0000 7ff2cc7cc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.754 INFO:teuthology.orchestra.run.trial160.stdout:{ 2026-02-26T03:57:50.754 INFO:teuthology.orchestra.run.trial160.stdout: "success": "" 2026-02-26T03:57:50.754 INFO:teuthology.orchestra.run.trial160.stdout:} 2026-02-26T03:57:50.762 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-26T03:57:50.841 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.834+0000 7f7ce9d7e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.843 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:50.839+0000 7f7ce9d7e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:50.957 INFO:teuthology.orchestra.run.trial160.stderr:instructed osd(s) 2 to deep-scrub 2026-02-26T03:57:50.967 INFO:tasks.ceph:Scrubbing osd.3 2026-02-26T03:57:50.967 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-26T03:57:51.048 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:51.039+0000 7fdaf141a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:51.050 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:51.046+0000 7fdaf141a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:51.061 INFO:teuthology.orchestra.run.trial160.stdout:{ 2026-02-26T03:57:51.061 INFO:teuthology.orchestra.run.trial160.stdout: "success": "" 2026-02-26T03:57:51.061 INFO:teuthology.orchestra.run.trial160.stdout:} 2026-02-26T03:57:51.068 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-26T03:57:51.147 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:51.140+0000 7fb556ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:51.150 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:51.146+0000 7fb556ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:51.260 INFO:teuthology.orchestra.run.trial160.stderr:instructed osd(s) 3 to deep-scrub 2026-02-26T03:57:51.271 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:57:51.348 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:51.341+0000 7fc776ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:51.350 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:57:51.346+0000 7fc776ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:57:51.461 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:57:51.462 INFO:teuthology.orchestra.run.trial160.stderr:dumped all 2026-02-26T03:57:51.470 INFO:teuthology.orchestra.run.trial160.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2026-02-26T03:57:49.741164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83897220,"kb_used_data":83897220,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125817916,"statfs":{"total":214748299264,"available":128837545984,"internally_reserved":0,"allocated":85910753280,"data_stored":85910753280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.124334"},"pg_stats":[{"pgid":"73.f","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.688332+0000","last_change":"2026-02-26T03:57:36.698466+0000","last_active":"2026-02-26T03:57:47.688332+0000","last_peered":"2026-02-26T03:57:47.688332+0000","last_clean":"2026-02-26T03:57:47.688332+0000","last_became_active":"2026-02-26T03:57:36.694282+0000","last_became_peered":"2026-02-26T03:57:36.694282+0000","last_unstale":"2026-02-26T03:57:47.688332+0000","last_undegraded":"2026-02-26T03:57:47.688332+0000","last_fullsized":"2026-02-26T03:57:47.688332+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.e","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.678023+0000","last_change":"2026-02-26T03:57:36.695055+0000","last_active":"2026-02-26T03:57:47.678023+0000","last_peered":"2026-02-26T03:57:47.678023+0000","last_clean":"2026-02-26T03:57:47.678023+0000","last_became_active":"2026-02-26T03:57:36.689826+0000","last_became_peered":"2026-02-26T03:57:36.689826+0000","last_unstale":"2026-02-26T03:57:47.678023+0000","last_undegraded":"2026-02-26T03:57:47.678023+0000","last_fullsized":"2026-02-26T03:57:47.678023+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.d","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.681678+0000","last_change":"2026-02-26T03:57:36.688656+0000","last_active":"2026-02-26T03:57:47.681678+0000","last_peered":"2026-02-26T03:57:47.681678+0000","last_clean":"2026-02-26T03:57:47.681678+0000","last_became_active":"2026-02-26T03:57:36.672224+0000","last_became_peered":"2026-02-26T03:57:36.672224+0000","last_unstale":"2026-02-26T03:57:47.681678+0000","last_undegraded":"2026-02-26T03:57:47.681678+0000","last_fullsized":"2026-02-26T03:57:47.681678+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.c","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.682658+0000","last_change":"2026-02-26T03:57:36.669418+0000","last_active":"2026-02-26T03:57:47.682658+0000","last_peered":"2026-02-26T03:57:47.682658+0000","last_clean":"2026-02-26T03:57:47.682658+0000","last_became_active":"2026-02-26T03:57:36.665056+0000","last_became_peered":"2026-02-26T03:57:36.665056+0000","last_unstale":"2026-02-26T03:57:47.682658+0000","last_undegraded":"2026-02-26T03:57:47.682658+0000","last_fullsized":"2026-02-26T03:57:47.682658+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.b","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.688411+0000","last_change":"2026-02-26T03:57:36.700898+0000","last_active":"2026-02-26T03:57:47.688411+0000","last_peered":"2026-02-26T03:57:47.688411+0000","last_clean":"2026-02-26T03:57:47.688411+0000","last_became_active":"2026-02-26T03:57:36.696446+0000","last_became_peered":"2026-02-26T03:57:36.696446+0000","last_unstale":"2026-02-26T03:57:47.688411+0000","last_undegraded":"2026-02-26T03:57:47.688411+0000","last_fullsized":"2026-02-26T03:57:47.688411+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.a","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.678597+0000","last_change":"2026-02-26T03:57:36.722333+0000","last_active":"2026-02-26T03:57:47.678597+0000","last_peered":"2026-02-26T03:57:47.678597+0000","last_clean":"2026-02-26T03:57:47.678597+0000","last_became_active":"2026-02-26T03:57:36.712454+0000","last_became_peered":"2026-02-26T03:57:36.712454+0000","last_unstale":"2026-02-26T03:57:47.678597+0000","last_undegraded":"2026-02-26T03:57:47.678597+0000","last_fullsized":"2026-02-26T03:57:47.678597+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.9","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.683819+0000","last_change":"2026-02-26T03:57:36.688198+0000","last_active":"2026-02-26T03:57:47.683819+0000","last_peered":"2026-02-26T03:57:47.683819+0000","last_clean":"2026-02-26T03:57:47.683819+0000","last_became_active":"2026-02-26T03:57:36.681925+0000","last_became_peered":"2026-02-26T03:57:36.681925+0000","last_unstale":"2026-02-26T03:57:47.683819+0000","last_undegraded":"2026-02-26T03:57:47.683819+0000","last_fullsized":"2026-02-26T03:57:47.683819+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.8","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.680564+0000","last_change":"2026-02-26T03:57:36.681425+0000","last_active":"2026-02-26T03:57:47.680564+0000","last_peered":"2026-02-26T03:57:47.680564+0000","last_clean":"2026-02-26T03:57:47.680564+0000","last_became_active":"2026-02-26T03:57:36.670570+0000","last_became_peered":"2026-02-26T03:57:36.670570+0000","last_unstale":"2026-02-26T03:57:47.680564+0000","last_undegraded":"2026-02-26T03:57:47.680564+0000","last_fullsized":"2026-02-26T03:57:47.680564+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.7","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.681426+0000","last_change":"2026-02-26T03:57:36.689553+0000","last_active":"2026-02-26T03:57:47.681426+0000","last_peered":"2026-02-26T03:57:47.681426+0000","last_clean":"2026-02-26T03:57:47.681426+0000","last_became_active":"2026-02-26T03:57:36.676014+0000","last_became_peered":"2026-02-26T03:57:36.676014+0000","last_unstale":"2026-02-26T03:57:47.681426+0000","last_undegraded":"2026-02-26T03:57:47.681426+0000","last_fullsized":"2026-02-26T03:57:47.681426+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.6","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.678663+0000","last_change":"2026-02-26T03:57:36.718085+0000","last_active":"2026-02-26T03:57:47.678663+0000","last_peered":"2026-02-26T03:57:47.678663+0000","last_clean":"2026-02-26T03:57:47.678663+0000","last_became_active":"2026-02-26T03:57:36.702612+0000","last_became_peered":"2026-02-26T03:57:36.702612+0000","last_unstale":"2026-02-26T03:57:47.678663+0000","last_undegraded":"2026-02-26T03:57:47.678663+0000","last_fullsized":"2026-02-26T03:57:47.678663+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.5","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.681882+0000","last_change":"2026-02-26T03:57:36.683624+0000","last_active":"2026-02-26T03:57:47.681882+0000","last_peered":"2026-02-26T03:57:47.681882+0000","last_clean":"2026-02-26T03:57:47.681882+0000","last_became_active":"2026-02-26T03:57:36.666470+0000","last_became_peered":"2026-02-26T03:57:36.666470+0000","last_unstale":"2026-02-26T03:57:47.681882+0000","last_undegraded":"2026-02-26T03:57:47.681882+0000","last_fullsized":"2026-02-26T03:57:47.681882+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.4","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.683738+0000","last_change":"2026-02-26T03:57:36.707220+0000","last_active":"2026-02-26T03:57:47.683738+0000","last_peered":"2026-02-26T03:57:47.683738+0000","last_clean":"2026-02-26T03:57:47.683738+0000","last_became_active":"2026-02-26T03:57:36.691808+0000","last_became_peered":"2026-02-26T03:57:36.691808+0000","last_unstale":"2026-02-26T03:57:47.683738+0000","last_undegraded":"2026-02-26T03:57:47.683738+0000","last_fullsized":"2026-02-26T03:57:47.683738+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.3","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.681945+0000","last_change":"2026-02-26T03:57:36.680547+0000","last_active":"2026-02-26T03:57:47.681945+0000","last_peered":"2026-02-26T03:57:47.681945+0000","last_clean":"2026-02-26T03:57:47.681945+0000","last_became_active":"2026-02-26T03:57:36.671742+0000","last_became_peered":"2026-02-26T03:57:36.671742+0000","last_unstale":"2026-02-26T03:57:47.681945+0000","last_undegraded":"2026-02-26T03:57:47.681945+0000","last_fullsized":"2026-02-26T03:57:47.681945+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.2","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.679299+0000","last_change":"2026-02-26T03:57:36.688778+0000","last_active":"2026-02-26T03:57:47.679299+0000","last_peered":"2026-02-26T03:57:47.679299+0000","last_clean":"2026-02-26T03:57:47.679299+0000","last_became_active":"2026-02-26T03:57:36.680023+0000","last_became_peered":"2026-02-26T03:57:36.680023+0000","last_unstale":"2026-02-26T03:57:47.679299+0000","last_undegraded":"2026-02-26T03:57:47.679299+0000","last_fullsized":"2026-02-26T03:57:47.679299+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.1","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.680579+0000","last_change":"2026-02-26T03:57:36.698512+0000","last_active":"2026-02-26T03:57:47.680579+0000","last_peered":"2026-02-26T03:57:47.680579+0000","last_clean":"2026-02-26T03:57:47.680579+0000","last_became_active":"2026-02-26T03:57:36.684373+0000","last_became_peered":"2026-02-26T03:57:36.684373+0000","last_unstale":"2026-02-26T03:57:47.680579+0000","last_undegraded":"2026-02-26T03:57:47.680579+0000","last_fullsized":"2026-02-26T03:57:47.680579+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.0","version":"0'0","reported_seq":29,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.687617+0000","last_change":"2026-02-26T03:57:36.699973+0000","last_active":"2026-02-26T03:57:47.687617+0000","last_peered":"2026-02-26T03:57:47.687617+0000","last_clean":"2026-02-26T03:57:47.687617+0000","last_became_active":"2026-02-26T03:57:36.687150+0000","last_became_peered":"2026-02-26T03:57:36.687150+0000","last_unstale":"2026-02-26T03:57:47.687617+0000","last_undegraded":"2026-02-26T03:57:47.687617+0000","last_fullsized":"2026-02-26T03:57:47.687617+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:35.582787+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":924,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.679072+0000","last_change":"2026-02-26T03:52:35.750083+0000","last_active":"2026-02-26T03:57:47.679072+0000","last_peered":"2026-02-26T03:57:47.679072+0000","last_clean":"2026-02-26T03:57:47.679072+0000","last_became_active":"2026-02-26T03:52:35.738890+0000","last_became_peered":"2026-02-26T03:52:35.738890+0000","last_unstale":"2026-02-26T03:57:47.679072+0000","last_undegraded":"2026-02-26T03:57:47.679072+0000","last_fullsized":"2026-02-26T03:57:47.679072+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:34.702059+0000","last_clean_scrub_stamp":"2026-02-26T03:52:34.702059+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":856,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.677249+0000","last_change":"2026-02-26T03:52:38.737043+0000","last_active":"2026-02-26T03:57:47.677249+0000","last_peered":"2026-02-26T03:57:47.677249+0000","last_clean":"2026-02-26T03:57:47.677249+0000","last_became_active":"2026-02-26T03:52:36.748803+0000","last_became_peered":"2026-02-26T03:52:36.748803+0000","last_unstale":"2026-02-26T03:57:47.677249+0000","last_undegraded":"2026-02-26T03:57:47.677249+0000","last_fullsized":"2026-02-26T03:57:47.677249+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.683160+0000","last_change":"2026-02-26T03:52:38.735698+0000","last_active":"2026-02-26T03:57:47.683160+0000","last_peered":"2026-02-26T03:57:47.683160+0000","last_clean":"2026-02-26T03:57:47.683160+0000","last_became_active":"2026-02-26T03:52:36.748522+0000","last_became_peered":"2026-02-26T03:52:36.748522+0000","last_unstale":"2026-02-26T03:57:47.683160+0000","last_undegraded":"2026-02-26T03:57:47.683160+0000","last_fullsized":"2026-02-26T03:57:47.683160+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.676706+0000","last_change":"2026-02-26T03:52:38.735296+0000","last_active":"2026-02-26T03:57:47.676706+0000","last_peered":"2026-02-26T03:57:47.676706+0000","last_clean":"2026-02-26T03:57:47.676706+0000","last_became_active":"2026-02-26T03:52:36.744177+0000","last_became_peered":"2026-02-26T03:52:36.744177+0000","last_unstale":"2026-02-26T03:57:47.676706+0000","last_undegraded":"2026-02-26T03:57:47.676706+0000","last_fullsized":"2026-02-26T03:57:47.676706+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":857,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.687932+0000","last_change":"2026-02-26T03:52:38.735604+0000","last_active":"2026-02-26T03:57:47.687932+0000","last_peered":"2026-02-26T03:57:47.687932+0000","last_clean":"2026-02-26T03:57:47.687932+0000","last_became_active":"2026-02-26T03:52:36.745315+0000","last_became_peered":"2026-02-26T03:52:36.745315+0000","last_unstale":"2026-02-26T03:57:47.687932+0000","last_undegraded":"2026-02-26T03:57:47.687932+0000","last_fullsized":"2026-02-26T03:57:47.687932+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.679925+0000","last_change":"2026-02-26T03:52:38.735590+0000","last_active":"2026-02-26T03:57:47.679925+0000","last_peered":"2026-02-26T03:57:47.679925+0000","last_clean":"2026-02-26T03:57:47.679925+0000","last_became_active":"2026-02-26T03:52:36.744870+0000","last_became_peered":"2026-02-26T03:52:36.744870+0000","last_unstale":"2026-02-26T03:57:47.679925+0000","last_undegraded":"2026-02-26T03:57:47.679925+0000","last_fullsized":"2026-02-26T03:57:47.679925+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.682202+0000","last_change":"2026-02-26T03:52:38.736040+0000","last_active":"2026-02-26T03:57:47.682202+0000","last_peered":"2026-02-26T03:57:47.682202+0000","last_clean":"2026-02-26T03:57:47.682202+0000","last_became_active":"2026-02-26T03:52:36.747323+0000","last_became_peered":"2026-02-26T03:52:36.747323+0000","last_unstale":"2026-02-26T03:57:47.682202+0000","last_undegraded":"2026-02-26T03:57:47.682202+0000","last_fullsized":"2026-02-26T03:57:47.682202+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.682556+0000","last_change":"2026-02-26T03:52:38.737272+0000","last_active":"2026-02-26T03:57:47.682556+0000","last_peered":"2026-02-26T03:57:47.682556+0000","last_clean":"2026-02-26T03:57:47.682556+0000","last_became_active":"2026-02-26T03:52:36.756413+0000","last_became_peered":"2026-02-26T03:52:36.756413+0000","last_unstale":"2026-02-26T03:57:47.682556+0000","last_undegraded":"2026-02-26T03:57:47.682556+0000","last_fullsized":"2026-02-26T03:57:47.682556+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":855,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:47.679175+0000","last_change":"2026-02-26T03:52:38.736727+0000","last_active":"2026-02-26T03:57:47.679175+0000","last_peered":"2026-02-26T03:57:47.679175+0000","last_clean":"2026-02-26T03:57:47.679175+0000","last_became_active":"2026-02-26T03:52:36.749872+0000","last_became_peered":"2026-02-26T03:52:36.749872+0000","last_unstale":"2026-02-26T03:57:47.679175+0000","last_undegraded":"2026-02-26T03:57:47.679175+0000","last_fullsized":"2026-02-26T03:57:47.679175+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:52:35.704991+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":73,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128837808128,"internally_reserved":0,"allocated":85910491136,"data_stored":85910491136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128837808128,"internally_reserved":0,"allocated":85910491136,"data_stored":85910491136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32209174528,"internally_reserved":0,"allocated":21477900288,"data_stored":21477900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974512,"kb_used_data":20974512,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454272,"statfs":{"total":53687074816,"available":32209174528,"internally_reserved":0,"allocated":21477900288,"data_stored":21477900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673274,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974024,"kb_used_data":20974024,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454760,"statfs":{"total":53687074816,"available":32209674240,"internally_reserved":0,"allocated":21477400576,"data_stored":21477400576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20973892,"kb_used_data":20973892,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454892,"statfs":{"total":53687074816,"available":32209809408,"internally_reserved":0,"allocated":21477265408,"data_stored":21477265408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673274,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974792,"kb_used_data":20974792,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31453992,"statfs":{"total":53687074816,"available":32208887808,"internally_reserved":0,"allocated":21478187008,"data_stored":21478187008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32209174528,"internally_reserved":0,"allocated":21477900288,"data_stored":21477900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32209068032,"internally_reserved":0,"allocated":21478006784,"data_stored":21478006784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32209809408,"internally_reserved":0,"allocated":21477265408,"data_stored":21477265408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32209756160,"internally_reserved":0,"allocated":21477318656,"data_stored":21477318656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32209174528,"internally_reserved":0,"allocated":21477900288,"data_stored":21477900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":0,"total":53687074816,"available":32209068032,"internally_reserved":0,"allocated":21478006784,"data_stored":21478006784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":1,"total":53687074816,"available":32209809408,"internally_reserved":0,"allocated":21477265408,"data_stored":21477265408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":2,"total":53687074816,"available":32209756160,"internally_reserved":0,"allocated":21477318656,"data_stored":21477318656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":3,"total":53687074816,"available":32209174528,"internally_reserved":0,"allocated":21477900288,"data_stored":21477900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:57:51.477 INFO:tasks.ceph:pgid 73.f last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.477 INFO:tasks.ceph:pgid 73.e last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.477 INFO:tasks.ceph:pgid 73.d last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.478 INFO:tasks.ceph:pgid 73.c last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.478 INFO:tasks.ceph:pgid 73.b last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.478 INFO:tasks.ceph:pgid 73.a last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.478 INFO:tasks.ceph:pgid 73.9 last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.478 INFO:tasks.ceph:pgid 73.8 last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.478 INFO:tasks.ceph:pgid 73.7 last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.478 INFO:tasks.ceph:pgid 73.6 last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.478 INFO:tasks.ceph:pgid 73.5 last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.478 INFO:tasks.ceph:pgid 73.4 last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.478 INFO:tasks.ceph:pgid 73.3 last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.478 INFO:tasks.ceph:pgid 73.2 last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.479 INFO:tasks.ceph:pgid 73.1 last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.479 INFO:tasks.ceph:pgid 73.0 last_scrub_stamp 2026-02-26T03:57:35.582787+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.479 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-26T03:52:34.702059+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=52, tm_sec=34, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.479 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-26T03:52:35.704991+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=52, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.479 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-26T03:52:35.704991+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=52, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.479 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-26T03:52:35.704991+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=52, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.479 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-26T03:52:35.704991+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=52, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.479 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-26T03:52:35.704991+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=52, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.479 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-26T03:52:35.704991+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=52, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.479 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-26T03:52:35.704991+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=52, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.479 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-26T03:52:35.704991+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=52, tm_sec=35, tm_wday=3, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=26, tm_hour=3, tm_min=57, tm_sec=49, tm_wday=3, tm_yday=57, tm_isdst=0) 2026-02-26T03:57:51.480 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-26T03:58:11.480 DEBUG:teuthology.orchestra.run.trial160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-26T03:58:11.564 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:58:11.557+0000 7effab5a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:58:11.566 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:58:11.562+0000 7effab5a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:58:11.679 INFO:teuthology.orchestra.run.trial160.stdout: 2026-02-26T03:58:11.680 INFO:teuthology.orchestra.run.trial160.stderr:dumped all 2026-02-26T03:58:11.690 INFO:teuthology.orchestra.run.trial160.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2026-02-26T03:58:09.744257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83896964,"kb_used_data":83896964,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125818172,"statfs":{"total":214748299264,"available":128837808128,"internally_reserved":0,"allocated":85910491136,"data_stored":85910491136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002076"},"pg_stats":[{"pgid":"73.f","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.960574+0000","last_change":"2026-02-26T03:57:50.975394+0000","last_active":"2026-02-26T03:57:50.960574+0000","last_peered":"2026-02-26T03:57:50.960574+0000","last_clean":"2026-02-26T03:57:50.960574+0000","last_became_active":"2026-02-26T03:57:36.694282+0000","last_became_peered":"2026-02-26T03:57:36.694282+0000","last_unstale":"2026-02-26T03:57:50.960574+0000","last_undegraded":"2026-02-26T03:57:50.960574+0000","last_fullsized":"2026-02-26T03:57:50.960574+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.975326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.975326+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.e","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.649886+0000","last_change":"2026-02-26T03:57:50.657428+0000","last_active":"2026-02-26T03:57:50.649886+0000","last_peered":"2026-02-26T03:57:50.649886+0000","last_clean":"2026-02-26T03:57:50.649886+0000","last_became_active":"2026-02-26T03:57:36.689826+0000","last_became_peered":"2026-02-26T03:57:36.689826+0000","last_unstale":"2026-02-26T03:57:50.649886+0000","last_undegraded":"2026-02-26T03:57:50.649886+0000","last_fullsized":"2026-02-26T03:57:50.649886+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.657321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.657321+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.d","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.959948+0000","last_change":"2026-02-26T03:57:50.978156+0000","last_active":"2026-02-26T03:57:50.959948+0000","last_peered":"2026-02-26T03:57:50.959948+0000","last_clean":"2026-02-26T03:57:50.959948+0000","last_became_active":"2026-02-26T03:57:36.672224+0000","last_became_peered":"2026-02-26T03:57:36.672224+0000","last_unstale":"2026-02-26T03:57:50.959948+0000","last_undegraded":"2026-02-26T03:57:50.959948+0000","last_fullsized":"2026-02-26T03:57:50.959948+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.978106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.978106+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.c","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.959598+0000","last_change":"2026-02-26T03:57:50.970976+0000","last_active":"2026-02-26T03:57:50.959598+0000","last_peered":"2026-02-26T03:57:50.959598+0000","last_clean":"2026-02-26T03:57:50.959598+0000","last_became_active":"2026-02-26T03:57:36.665056+0000","last_became_peered":"2026-02-26T03:57:36.665056+0000","last_unstale":"2026-02-26T03:57:50.959598+0000","last_undegraded":"2026-02-26T03:57:50.959598+0000","last_fullsized":"2026-02-26T03:57:50.959598+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.970892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.970892+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.b","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.958292+0000","last_change":"2026-02-26T03:57:50.967975+0000","last_active":"2026-02-26T03:57:50.958292+0000","last_peered":"2026-02-26T03:57:50.958292+0000","last_clean":"2026-02-26T03:57:50.958292+0000","last_became_active":"2026-02-26T03:57:36.696446+0000","last_became_peered":"2026-02-26T03:57:36.696446+0000","last_unstale":"2026-02-26T03:57:50.958292+0000","last_undegraded":"2026-02-26T03:57:50.958292+0000","last_fullsized":"2026-02-26T03:57:50.958292+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.967877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.967877+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.a","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.649993+0000","last_change":"2026-02-26T03:57:50.658391+0000","last_active":"2026-02-26T03:57:50.649993+0000","last_peered":"2026-02-26T03:57:50.649993+0000","last_clean":"2026-02-26T03:57:50.649993+0000","last_became_active":"2026-02-26T03:57:36.712454+0000","last_became_peered":"2026-02-26T03:57:36.712454+0000","last_unstale":"2026-02-26T03:57:50.649993+0000","last_undegraded":"2026-02-26T03:57:50.649993+0000","last_fullsized":"2026-02-26T03:57:50.649993+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.658333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.658333+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.9","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:51.261359+0000","last_change":"2026-02-26T03:57:51.273548+0000","last_active":"2026-02-26T03:57:51.261359+0000","last_peered":"2026-02-26T03:57:51.261359+0000","last_clean":"2026-02-26T03:57:51.261359+0000","last_became_active":"2026-02-26T03:57:36.681925+0000","last_became_peered":"2026-02-26T03:57:36.681925+0000","last_unstale":"2026-02-26T03:57:51.261359+0000","last_undegraded":"2026-02-26T03:57:51.261359+0000","last_fullsized":"2026-02-26T03:57:51.261359+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:51.273476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:51.273476+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.8","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.343893+0000","last_change":"2026-02-26T03:57:50.351519+0000","last_active":"2026-02-26T03:57:50.343893+0000","last_peered":"2026-02-26T03:57:50.343893+0000","last_clean":"2026-02-26T03:57:50.343893+0000","last_became_active":"2026-02-26T03:57:36.670570+0000","last_became_peered":"2026-02-26T03:57:36.670570+0000","last_unstale":"2026-02-26T03:57:50.343893+0000","last_undegraded":"2026-02-26T03:57:50.343893+0000","last_fullsized":"2026-02-26T03:57:50.343893+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.351465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.351465+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.7","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:51.260280+0000","last_change":"2026-02-26T03:57:51.274273+0000","last_active":"2026-02-26T03:57:51.260280+0000","last_peered":"2026-02-26T03:57:51.260280+0000","last_clean":"2026-02-26T03:57:51.260280+0000","last_became_active":"2026-02-26T03:57:36.676014+0000","last_became_peered":"2026-02-26T03:57:36.676014+0000","last_unstale":"2026-02-26T03:57:51.260280+0000","last_undegraded":"2026-02-26T03:57:51.260280+0000","last_fullsized":"2026-02-26T03:57:51.260280+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:51.274222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:51.274222+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.6","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.957749+0000","last_change":"2026-02-26T03:57:50.977161+0000","last_active":"2026-02-26T03:57:50.957749+0000","last_peered":"2026-02-26T03:57:50.957749+0000","last_clean":"2026-02-26T03:57:50.957749+0000","last_became_active":"2026-02-26T03:57:36.702612+0000","last_became_peered":"2026-02-26T03:57:36.702612+0000","last_unstale":"2026-02-26T03:57:50.957749+0000","last_undegraded":"2026-02-26T03:57:50.957749+0000","last_fullsized":"2026-02-26T03:57:50.957749+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.977110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.977110+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.5","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.957050+0000","last_change":"2026-02-26T03:57:50.977888+0000","last_active":"2026-02-26T03:57:50.957050+0000","last_peered":"2026-02-26T03:57:50.957050+0000","last_clean":"2026-02-26T03:57:50.957050+0000","last_became_active":"2026-02-26T03:57:36.666470+0000","last_became_peered":"2026-02-26T03:57:36.666470+0000","last_unstale":"2026-02-26T03:57:50.957050+0000","last_undegraded":"2026-02-26T03:57:50.957050+0000","last_fullsized":"2026-02-26T03:57:50.957050+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.977839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.977839+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"73.4","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.343485+0000","last_change":"2026-02-26T03:57:50.351209+0000","last_active":"2026-02-26T03:57:50.343485+0000","last_peered":"2026-02-26T03:57:50.343485+0000","last_clean":"2026-02-26T03:57:50.343485+0000","last_became_active":"2026-02-26T03:57:36.691808+0000","last_became_peered":"2026-02-26T03:57:36.691808+0000","last_unstale":"2026-02-26T03:57:50.343485+0000","last_undegraded":"2026-02-26T03:57:50.343485+0000","last_fullsized":"2026-02-26T03:57:50.343485+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.351150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.351150+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"73.3","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:51.261077+0000","last_change":"2026-02-26T03:57:51.270387+0000","last_active":"2026-02-26T03:57:51.261077+0000","last_peered":"2026-02-26T03:57:51.261077+0000","last_clean":"2026-02-26T03:57:51.261077+0000","last_became_active":"2026-02-26T03:57:36.671742+0000","last_became_peered":"2026-02-26T03:57:36.671742+0000","last_unstale":"2026-02-26T03:57:51.261077+0000","last_undegraded":"2026-02-26T03:57:51.261077+0000","last_fullsized":"2026-02-26T03:57:51.261077+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:51.270316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:51.270316+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.2","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.649421+0000","last_change":"2026-02-26T03:57:50.658024+0000","last_active":"2026-02-26T03:57:50.649421+0000","last_peered":"2026-02-26T03:57:50.649421+0000","last_clean":"2026-02-26T03:57:50.649421+0000","last_became_active":"2026-02-26T03:57:36.680023+0000","last_became_peered":"2026-02-26T03:57:36.680023+0000","last_unstale":"2026-02-26T03:57:50.649421+0000","last_undegraded":"2026-02-26T03:57:50.649421+0000","last_fullsized":"2026-02-26T03:57:50.649421+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.657965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.657965+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"73.1","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:51.260333+0000","last_change":"2026-02-26T03:57:51.269936+0000","last_active":"2026-02-26T03:57:51.260333+0000","last_peered":"2026-02-26T03:57:51.260333+0000","last_clean":"2026-02-26T03:57:51.260333+0000","last_became_active":"2026-02-26T03:57:36.684373+0000","last_became_peered":"2026-02-26T03:57:36.684373+0000","last_unstale":"2026-02-26T03:57:51.260333+0000","last_undegraded":"2026-02-26T03:57:51.260333+0000","last_fullsized":"2026-02-26T03:57:51.260333+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:51.269851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:51.269851+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"73.0","version":"0'0","reported_seq":31,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.344047+0000","last_change":"2026-02-26T03:57:50.351561+0000","last_active":"2026-02-26T03:57:50.344047+0000","last_peered":"2026-02-26T03:57:50.344047+0000","last_clean":"2026-02-26T03:57:50.344047+0000","last_became_active":"2026-02-26T03:57:36.687150+0000","last_became_peered":"2026-02-26T03:57:36.687150+0000","last_unstale":"2026-02-26T03:57:50.344047+0000","last_undegraded":"2026-02-26T03:57:50.344047+0000","last_fullsized":"2026-02-26T03:57:50.344047+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.351508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.351508+0000","last_clean_scrub_stamp":"2026-02-26T03:57:35.582787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":926,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:51.258706+0000","last_change":"2026-02-26T03:57:51.284061+0000","last_active":"2026-02-26T03:57:51.258706+0000","last_peered":"2026-02-26T03:57:51.258706+0000","last_clean":"2026-02-26T03:57:51.258706+0000","last_became_active":"2026-02-26T03:52:35.738890+0000","last_became_peered":"2026-02-26T03:52:35.738890+0000","last_unstale":"2026-02-26T03:57:51.258706+0000","last_undegraded":"2026-02-26T03:57:51.258706+0000","last_fullsized":"2026-02-26T03:57:51.258706+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'52","last_scrub_stamp":"2026-02-26T03:57:51.283974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:51.283974+0000","last_clean_scrub_stamp":"2026-02-26T03:52:34.702059+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":858,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.648881+0000","last_change":"2026-02-26T03:57:50.660828+0000","last_active":"2026-02-26T03:57:50.648881+0000","last_peered":"2026-02-26T03:57:50.648881+0000","last_clean":"2026-02-26T03:57:50.648881+0000","last_became_active":"2026-02-26T03:52:36.748803+0000","last_became_peered":"2026-02-26T03:52:36.748803+0000","last_unstale":"2026-02-26T03:57:50.648881+0000","last_undegraded":"2026-02-26T03:57:50.648881+0000","last_fullsized":"2026-02-26T03:57:50.648881+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"13'1","last_scrub_stamp":"2026-02-26T03:57:50.660761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.660761+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":857,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.343298+0000","last_change":"2026-02-26T03:57:50.350658+0000","last_active":"2026-02-26T03:57:50.343298+0000","last_peered":"2026-02-26T03:57:50.343298+0000","last_clean":"2026-02-26T03:57:50.343298+0000","last_became_active":"2026-02-26T03:52:36.748522+0000","last_became_peered":"2026-02-26T03:52:36.748522+0000","last_unstale":"2026-02-26T03:57:50.343298+0000","last_undegraded":"2026-02-26T03:57:50.343298+0000","last_fullsized":"2026-02-26T03:57:50.343298+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.350587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.350587+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":857,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.648183+0000","last_change":"2026-02-26T03:57:50.656718+0000","last_active":"2026-02-26T03:57:50.648183+0000","last_peered":"2026-02-26T03:57:50.648183+0000","last_clean":"2026-02-26T03:57:50.648183+0000","last_became_active":"2026-02-26T03:52:36.744177+0000","last_became_peered":"2026-02-26T03:52:36.744177+0000","last_unstale":"2026-02-26T03:57:50.648183+0000","last_undegraded":"2026-02-26T03:57:50.648183+0000","last_fullsized":"2026-02-26T03:57:50.648183+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.656654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.656654+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":859,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.343304+0000","last_change":"2026-02-26T03:57:50.354537+0000","last_active":"2026-02-26T03:57:50.343304+0000","last_peered":"2026-02-26T03:57:50.343304+0000","last_clean":"2026-02-26T03:57:50.343304+0000","last_became_active":"2026-02-26T03:52:36.745315+0000","last_became_peered":"2026-02-26T03:52:36.745315+0000","last_unstale":"2026-02-26T03:57:50.343304+0000","last_undegraded":"2026-02-26T03:57:50.343304+0000","last_fullsized":"2026-02-26T03:57:50.343304+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2026-02-26T03:57:50.354458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.354458+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":857,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.955501+0000","last_change":"2026-02-26T03:57:50.976475+0000","last_active":"2026-02-26T03:57:50.955501+0000","last_peered":"2026-02-26T03:57:50.955501+0000","last_clean":"2026-02-26T03:57:50.955501+0000","last_became_active":"2026-02-26T03:52:36.744870+0000","last_became_peered":"2026-02-26T03:52:36.744870+0000","last_unstale":"2026-02-26T03:57:50.955501+0000","last_undegraded":"2026-02-26T03:57:50.955501+0000","last_fullsized":"2026-02-26T03:57:50.955501+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.976427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.976427+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":857,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:51.259264+0000","last_change":"2026-02-26T03:57:51.269310+0000","last_active":"2026-02-26T03:57:51.259264+0000","last_peered":"2026-02-26T03:57:51.259264+0000","last_clean":"2026-02-26T03:57:51.259264+0000","last_became_active":"2026-02-26T03:52:36.747323+0000","last_became_peered":"2026-02-26T03:52:36.747323+0000","last_unstale":"2026-02-26T03:57:51.259264+0000","last_undegraded":"2026-02-26T03:57:51.259264+0000","last_fullsized":"2026-02-26T03:57:51.259264+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:51.269254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:51.269254+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":857,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.956520+0000","last_change":"2026-02-26T03:57:50.973730+0000","last_active":"2026-02-26T03:57:50.956520+0000","last_peered":"2026-02-26T03:57:50.956520+0000","last_clean":"2026-02-26T03:57:50.956520+0000","last_became_active":"2026-02-26T03:52:36.756413+0000","last_became_peered":"2026-02-26T03:52:36.756413+0000","last_unstale":"2026-02-26T03:57:50.956520+0000","last_undegraded":"2026-02-26T03:57:50.956520+0000","last_fullsized":"2026-02-26T03:57:50.956520+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.973649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.973649+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":857,"reported_epoch":295,"state":"active+clean","last_fresh":"2026-02-26T03:57:50.649415+0000","last_change":"2026-02-26T03:57:50.655858+0000","last_active":"2026-02-26T03:57:50.649415+0000","last_peered":"2026-02-26T03:57:50.649415+0000","last_clean":"2026-02-26T03:57:50.649415+0000","last_became_active":"2026-02-26T03:52:36.749872+0000","last_became_peered":"2026-02-26T03:52:36.749872+0000","last_unstale":"2026-02-26T03:57:50.649415+0000","last_undegraded":"2026-02-26T03:57:50.649415+0000","last_fullsized":"2026-02-26T03:57:50.649415+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-26T03:57:50.655750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-26T03:57:50.655750+0000","last_clean_scrub_stamp":"2026-02-26T03:52:35.704991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":73,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128837808128,"internally_reserved":0,"allocated":85910491136,"data_stored":85910491136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214748299264,"available":128837808128,"internally_reserved":0,"allocated":85910491136,"data_stored":85910491136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32209174528,"internally_reserved":0,"allocated":21477900288,"data_stored":21477900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974512,"kb_used_data":20974512,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454272,"statfs":{"total":53687074816,"available":32209174528,"internally_reserved":0,"allocated":21477900288,"data_stored":21477900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673294,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20973944,"kb_used_data":20973944,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454840,"statfs":{"total":53687074816,"available":32209756160,"internally_reserved":0,"allocated":21477318656,"data_stored":21477318656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20973892,"kb_used_data":20973892,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454892,"statfs":{"total":53687074816,"available":32209809408,"internally_reserved":0,"allocated":21477265408,"data_stored":21477265408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949673294,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974616,"kb_used_data":20974616,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454168,"statfs":{"total":53687074816,"available":32209068032,"internally_reserved":0,"allocated":21478006784,"data_stored":21478006784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32209174528,"internally_reserved":0,"allocated":21477900288,"data_stored":21477900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32209068032,"internally_reserved":0,"allocated":21478006784,"data_stored":21478006784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32209809408,"internally_reserved":0,"allocated":21477265408,"data_stored":21477265408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32209756160,"internally_reserved":0,"allocated":21477318656,"data_stored":21477318656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32209174528,"internally_reserved":0,"allocated":21477900288,"data_stored":21477900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":0,"total":53687074816,"available":32209068032,"internally_reserved":0,"allocated":21478006784,"data_stored":21478006784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":1,"total":53687074816,"available":32209809408,"internally_reserved":0,"allocated":21477265408,"data_stored":21477265408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":2,"total":53687074816,"available":32209756160,"internally_reserved":0,"allocated":21477318656,"data_stored":21477318656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":73,"osd":3,"total":53687074816,"available":32209174528,"internally_reserved":0,"allocated":21477900288,"data_stored":21477900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-26T03:58:11.693 DEBUG:teuthology.orchestra.run.trial160:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-26T03:58:11.768 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:58:11.760+0000 7f77a485a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:58:11.770 INFO:teuthology.orchestra.run.trial160.stderr:2026-02-26T03:58:11.766+0000 7f77a485a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-26T03:58:11.897 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-26T03:58:11.897 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-26T03:58:11.898 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-26T03:58:11.898 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:58:11.958 INFO:tasks.ceph.osd.0:Stopped 2026-02-26T03:58:11.958 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-26T03:58:11.959 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:58:12.028 INFO:tasks.ceph.osd.1:Stopped 2026-02-26T03:58:12.028 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-26T03:58:12.029 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:58:12.104 INFO:tasks.ceph.osd.2:Stopped 2026-02-26T03:58:12.104 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-26T03:58:12.104 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:58:12.168 INFO:tasks.ceph.osd.3:Stopped 2026-02-26T03:58:12.168 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-26T03:58:12.168 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-26T03:58:12.168 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:58:12.200 INFO:tasks.ceph.mgr.x:Stopped 2026-02-26T03:58:12.200 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-26T03:58:12.200 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-26T03:58:12.200 INFO:teuthology.orchestra.run:waiting for 300 2026-02-26T03:58:12.213 INFO:tasks.ceph.mon.a:Stopped 2026-02-26T03:58:12.213 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-26T03:58:12.213 DEBUG:teuthology.orchestra.run.trial160:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v 'but it is still running' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSDMAP_FLAGS\)' | grep -E -v '\(PG_' | grep -E -v '\(OSD_' | grep -E -v '\(OBJECT_' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2026-02-26T03:58:12.279 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial160.front.sepia.ceph.com 2026-02-26T03:58:12.279 DEBUG:teuthology.orchestra.run.trial160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-26T03:58:12.346 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial160.front.sepia.ceph.com 2026-02-26T03:58:12.346 DEBUG:teuthology.orchestra.run.trial160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-26T03:58:12.413 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial195.front.sepia.ceph.com 2026-02-26T03:58:12.413 DEBUG:teuthology.orchestra.run.trial195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-26T03:58:12.452 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial195.front.sepia.ceph.com 2026-02-26T03:58:12.453 DEBUG:teuthology.orchestra.run.trial195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-26T03:58:12.527 INFO:tasks.ceph:Archiving mon data... 2026-02-26T03:58:12.529 DEBUG:teuthology.misc:Transferring archived files from trial160:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71262/data/mon.a.tgz 2026-02-26T03:58:12.529 DEBUG:teuthology.orchestra.run.trial160:> mktemp 2026-02-26T03:58:12.547 INFO:teuthology.orchestra.run.trial160.stdout:/tmp/tmp.Fg1CoSCQCJ 2026-02-26T03:58:12.548 DEBUG:teuthology.orchestra.run.trial160:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Fg1CoSCQCJ 2026-02-26T03:58:12.661 DEBUG:teuthology.orchestra.run.trial160:> sudo chmod 0666 /tmp/tmp.Fg1CoSCQCJ 2026-02-26T03:58:12.743 DEBUG:teuthology.orchestra.remote:trial160:/tmp/tmp.Fg1CoSCQCJ is 383KB 2026-02-26T03:58:12.837 DEBUG:teuthology.orchestra.run.trial160:> rm -fr /tmp/tmp.Fg1CoSCQCJ 2026-02-26T03:58:12.855 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-26T03:58:12.855 DEBUG:teuthology.orchestra.run.trial160:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-26T03:58:12.897 DEBUG:teuthology.orchestra.run.trial195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-26T03:58:12.975 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-26T03:58:12.991 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-26T03:58:12.991 INFO:tasks.ceph:Archiving crash dumps... 2026-02-26T03:58:12.993 DEBUG:teuthology.misc:Transferring archived files from trial160:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71262/remote/trial160/crash 2026-02-26T03:58:12.995 DEBUG:teuthology.orchestra.run.trial160:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-26T03:58:13.040 DEBUG:teuthology.misc:Transferring archived files from trial195:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71262/remote/trial195/crash 2026-02-26T03:58:13.041 DEBUG:teuthology.orchestra.run.trial195:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-26T03:58:13.066 INFO:tasks.ceph:Compressing logs... 2026-02-26T03:58:13.066 DEBUG:teuthology.orchestra.run.trial160:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-26T03:58:13.085 DEBUG:teuthology.orchestra.run.trial195:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-26T03:58:13.107 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38105.log 2026-02-26T03:58:13.121 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-26T03:58:13.121 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41001.log 2026-02-26T03:58:13.122 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38105.log.gz 2026-02-26T03:58:13.122 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-26T03:58:13.122 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.41001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40658.log 2026-02-26T03:58:13.122 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41001.log.gz 2026-02-26T03:58:13.122 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38814.log 2026-02-26T03:58:13.122 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.40658.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.36168.log 2026-02-26T03:58:13.122 INFO:teuthology.orchestra.run.trial160.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40658.log.gz 2026-02-26T03:58:13.122 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41050.log 2026-02-26T03:58:13.122 INFO:teuthology.orchestra.run.trial160.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.38814.log.gz 2026-02-26T03:58:13.122 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph.tmp-client.admin.36168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41113.log 2026-02-26T03:58:13.122 INFO:teuthology.orchestra.run.trial160.stderr: 66.9% -- replaced with /var/log/ceph/ceph.tmp-client.admin.36168.log.gz 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.41050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40511.log 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41050.log.gz 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.41113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38272.log 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41113.log.gz 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.40511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37755.log 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40511.log.gz 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37927.log 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38272.log.gz 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37829.log 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37755.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.37755.log.gz 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37927.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37927.log.gz -5 2026-02-26T03:58:13.123 INFO:teuthology.orchestra.run.trial160.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37230.log 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40609.log 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37829.log.gz 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37230.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37230.log.gz 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37878.log 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.40609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38274.log 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40609.log.gz 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37878.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-client.admin.37597.log -- replaced with /var/log/ceph/ceph-client.admin.37878.log.gz 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr: 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40952.log 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38274.log.gz 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40462.log 2026-02-26T03:58:13.124 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37597.log.gz 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.40952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38417.log 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40952.log.gz 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.40462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38103.log 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40462.log.gz 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40854.log 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38417.log.gz 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38103.log: gzip 50.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.38103.log.gz 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40560.log 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.40854.log: gzip 50.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.40854.log.gz --verbose 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr: -- /var/log/ceph/ceph-client.admin.38789.log 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.40560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38275.log 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40560.log.gz 2026-02-26T03:58:13.125 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40903.log 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38789.log.gz 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38275.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38275.log.gz 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.40903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37363.log 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40903.log.gz 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40707.log 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37363.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37363.log.gz 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr: 88.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.40707.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40707.log.gz 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38106.log 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37316.log 2026-02-26T03:58:13.126 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38466.log 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38106.log.gz 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38104.log 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37730.log 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38466.log.gz 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37976.log 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38104.log.gz 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37730.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37730.log.gz 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38515.log 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37976.log.gz 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38302.log 2026-02-26T03:58:13.127 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.38515.log: gzip -5 --verbose -- 50.4% /var/log/ceph/ceph-client.admin.37571.log -- replaced with /var/log/ceph/ceph-client.admin.38515.log.gz 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr: 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.38302.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38302.log.gz 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40756.log 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41160.log 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37571.log: /var/log/ceph/ceph-client.admin.40756.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37571.log.gz 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40805.log 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40756.log.gz 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37659.log 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.41160.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41160.log.gz 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.40805.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40805.log.gz 2026-02-26T03:58:13.128 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/ceph-client.admin.37659.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37659.log.gz 2026-02-26T03:58:13.137 INFO:teuthology.orchestra.run.trial195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-26T03:58:13.139 INFO:teuthology.orchestra.run.trial195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-26T03:58:13.139 INFO:teuthology.orchestra.run.trial195.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37450.log 2026-02-26T03:58:13.139 INFO:teuthology.orchestra.run.trial195.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37555.log 2026-02-26T03:58:13.140 INFO:teuthology.orchestra.run.trial195.stderr:/var/log/ceph/ceph-client.admin.37450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37403.log 2026-02-26T03:58:13.140 INFO:teuthology.orchestra.run.trial195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37450.log.gz 2026-02-26T03:58:13.140 INFO:teuthology.orchestra.run.trial195.stderr:/var/log/ceph/ceph-client.admin.37555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37502.log 2026-02-26T03:58:13.140 INFO:teuthology.orchestra.run.trial195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37555.log.gz 2026-02-26T03:58:13.140 INFO:teuthology.orchestra.run.trial195.stderr:/var/log/ceph/ceph-client.admin.37403.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-26T03:58:13.140 INFO:teuthology.orchestra.run.trial195.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37403.log.gz 2026-02-26T03:58:13.140 INFO:teuthology.orchestra.run.trial195.stderr:/var/log/ceph/ceph-client.admin.37502.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37502.log.gz 2026-02-26T03:58:13.164 INFO:teuthology.orchestra.run.trial195.stderr:/var/log/ceph/ceph-mgr.x.log: 90.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-26T03:58:13.284 INFO:teuthology.orchestra.run.trial160.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-26T03:58:15.501 INFO:teuthology.orchestra.run.trial160.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-26T03:58:15.591 INFO:teuthology.orchestra.run.trial195.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-26T03:58:15.596 INFO:teuthology.orchestra.run.trial195.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-26T03:58:15.598 INFO:teuthology.orchestra.run.trial195.stderr: 2026-02-26T03:58:15.598 INFO:teuthology.orchestra.run.trial195.stderr:real 0m2.469s 2026-02-26T03:58:15.598 INFO:teuthology.orchestra.run.trial195.stderr:user 0m4.674s 2026-02-26T03:58:15.598 INFO:teuthology.orchestra.run.trial195.stderr:sys 0m0.275s 2026-02-26T03:58:15.718 INFO:teuthology.orchestra.run.trial160.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-26T03:58:15.720 INFO:teuthology.orchestra.run.trial160.stderr: 2026-02-26T03:58:15.720 INFO:teuthology.orchestra.run.trial160.stderr:real 0m2.622s 2026-02-26T03:58:15.720 INFO:teuthology.orchestra.run.trial160.stderr:user 0m4.918s 2026-02-26T03:58:15.720 INFO:teuthology.orchestra.run.trial160.stderr:sys 0m0.292s 2026-02-26T03:58:15.720 INFO:tasks.ceph:Archiving logs... 2026-02-26T03:58:15.721 DEBUG:teuthology.misc:Transferring archived files from trial160:/var/log/ceph to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71262/remote/trial160/log 2026-02-26T03:58:15.722 DEBUG:teuthology.orchestra.run.trial160:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-26T03:58:16.214 DEBUG:teuthology.misc:Transferring archived files from trial195:/var/log/ceph to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71262/remote/trial195/log 2026-02-26T03:58:16.214 DEBUG:teuthology.orchestra.run.trial195:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-26T03:58:16.628 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-26T03:58:16.762 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-26T03:58:16.791 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-26T03:58:16.792 DEBUG:teuthology.orchestra.run.trial160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-26T03:58:16.793 DEBUG:teuthology.orchestra.run.trial195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-26T03:58:16.820 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-26T03:58:16.831 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-26T03:58:16.831 DEBUG:teuthology.orchestra.run.trial160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-26T03:58:16.859 DEBUG:teuthology.orchestra.run.trial195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-26T03:58:16.872 INFO:teuthology.orchestra.run.trial160.stderr:bash: line 1: ntpq: command not found 2026-02-26T03:58:16.875 INFO:teuthology.orchestra.run.trial195.stderr:bash: line 1: ntpq: command not found 2026-02-26T03:58:16.893 INFO:teuthology.orchestra.run.trial160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-26T03:58:16.894 INFO:teuthology.orchestra.run.trial160.stdout:=============================================================================== 2026-02-26T03:58:16.894 INFO:teuthology.orchestra.run.trial160.stdout:^- y.ns.gin.ntt.net 2 6 377 55 -307us[ -78us] +/- 53ms 2026-02-26T03:58:16.894 INFO:teuthology.orchestra.run.trial160.stdout:^- linode1.ernest-doss.org 3 6 377 56 +187us[ +416us] +/- 78ms 2026-02-26T03:58:16.894 INFO:teuthology.orchestra.run.trial160.stdout:^* rn-02.koehn.com 2 6 377 54 +1089us[+1318us] +/- 13ms 2026-02-26T03:58:16.894 INFO:teuthology.orchestra.run.trial160.stdout:^- lax1.us.ntp.li 2 6 377 56 -124us[ +104us] +/- 35ms 2026-02-26T03:58:16.912 INFO:teuthology.orchestra.run.trial195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-26T03:58:16.912 INFO:teuthology.orchestra.run.trial195.stdout:=============================================================================== 2026-02-26T03:58:16.912 INFO:teuthology.orchestra.run.trial195.stdout:^* 96-19-94-82.cpe.sparklig> 1 6 377 55 -93us[ -376us] +/- 32ms 2026-02-26T03:58:16.912 INFO:teuthology.orchestra.run.trial195.stdout:^+ 69.9.131.124 2 6 377 55 +7266us[+7266us] +/- 69ms 2026-02-26T03:58:16.913 INFO:teuthology.orchestra.run.trial195.stdout:^+ 69.176.84.38 2 6 377 54 +996us[ +996us] +/- 35ms 2026-02-26T03:58:16.913 INFO:teuthology.orchestra.run.trial195.stdout:^+ 170-187-147-56.ip.linode> 2 6 377 54 -6398us[-6398us] +/- 85ms 2026-02-26T03:58:16.914 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-26T03:58:16.931 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-26T03:58:16.932 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-26T03:58:16.948 DEBUG:teuthology.orchestra.run.trial160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-26T03:58:17.000 DEBUG:teuthology.orchestra.run.trial195:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-26T03:58:17.055 DEBUG:teuthology.orchestra.run.trial160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-26T03:58:17.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:58:17.083 DEBUG:teuthology.orchestra.run.trial195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-26T03:58:17.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:58:17.109 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-26T03:58:17.125 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-26T03:58:17.139 INFO:teuthology.task.internal:Duration was 760.336706 seconds 2026-02-26T03:58:17.140 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-26T03:58:17.154 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-26T03:58:17.155 DEBUG:teuthology.orchestra.run.trial160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-26T03:58:17.157 DEBUG:teuthology.orchestra.run.trial195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-26T03:58:17.192 INFO:teuthology.orchestra.run.trial160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-26T03:58:17.194 INFO:teuthology.orchestra.run.trial195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-26T03:58:17.591 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-26T03:58:17.591 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial160.front.sepia.ceph.com 2026-02-26T03:58:17.591 DEBUG:teuthology.orchestra.run.trial160:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-26T03:58:17.650 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial195.front.sepia.ceph.com 2026-02-26T03:58:17.650 DEBUG:teuthology.orchestra.run.trial195:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-26T03:58:17.669 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-26T03:58:17.669 DEBUG:teuthology.orchestra.run.trial160:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-26T03:58:17.691 DEBUG:teuthology.orchestra.run.trial195:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-26T03:58:17.771 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-26T03:58:17.771 DEBUG:teuthology.orchestra.run.trial160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-26T03:58:17.774 DEBUG:teuthology.orchestra.run.trial195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-26T03:58:17.794 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:58:17.795 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:58:17.795 INFO:teuthology.orchestra.run.trial160.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-26T03:58:17.795 INFO:teuthology.orchestra.run.trial160.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-26T03:58:17.795 INFO:teuthology.orchestra.run.trial160.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-26T03:58:17.797 INFO:teuthology.orchestra.run.trial195.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:58:17.798 INFO:teuthology.orchestra.run.trial195.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:58:17.798 INFO:teuthology.orchestra.run.trial195.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-26T03:58:17.798 INFO:teuthology.orchestra.run.trial195.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-26T03:58:17.798 INFO:teuthology.orchestra.run.trial195.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-26T03:58:17.804 INFO:teuthology.orchestra.run.trial160.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-26T03:58:17.807 INFO:teuthology.orchestra.run.trial195.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.3% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-26T03:58:17.809 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-26T03:58:17.825 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-26T03:58:17.826 DEBUG:teuthology.orchestra.run.trial160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-26T03:58:17.871 DEBUG:teuthology.orchestra.run.trial195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-26T03:58:17.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-26T03:58:17.913 DEBUG:teuthology.orchestra.run.trial160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:58:17.916 DEBUG:teuthology.orchestra.run.trial195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:58:17.938 INFO:teuthology.orchestra.run.trial160.stdout:kernel.core_pattern = core 2026-02-26T03:58:17.960 INFO:teuthology.orchestra.run.trial195.stdout:kernel.core_pattern = core 2026-02-26T03:58:17.972 DEBUG:teuthology.orchestra.run.trial160:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:58:18.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:58:18.006 DEBUG:teuthology.orchestra.run.trial195:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:58:18.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:58:18.028 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-26T03:58:18.046 INFO:teuthology.task.internal:Transferring archived files... 2026-02-26T03:58:18.047 DEBUG:teuthology.misc:Transferring archived files from trial160:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71262/remote/trial160 2026-02-26T03:58:18.047 DEBUG:teuthology.orchestra.run.trial160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-26T03:58:18.083 DEBUG:teuthology.misc:Transferring archived files from trial195:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71262/remote/trial195 2026-02-26T03:58:18.084 DEBUG:teuthology.orchestra.run.trial195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-26T03:58:18.120 INFO:teuthology.task.internal:Removing archive directory... 2026-02-26T03:58:18.120 DEBUG:teuthology.orchestra.run.trial160:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-26T03:58:18.123 DEBUG:teuthology.orchestra.run.trial195:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-26T03:58:18.168 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-26T03:58:18.187 INFO:teuthology.task.internal:Not uploading archives. 2026-02-26T03:58:18.187 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-26T03:58:18.204 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-26T03:58:18.204 DEBUG:teuthology.orchestra.run.trial160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-26T03:58:18.207 DEBUG:teuthology.orchestra.run.trial195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-26T03:58:18.222 INFO:teuthology.orchestra.run.trial160.stdout: 917521 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 03:58 /home/ubuntu/cephtest 2026-02-26T03:58:18.224 INFO:teuthology.orchestra.run.trial195.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 03:58 /home/ubuntu/cephtest 2026-02-26T03:58:18.225 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-26T03:58:18.243 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-26T03:58:18.282 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/rados_python} duration: 760.3367059230804 flavor: debug owner: scheduled_shraddhaag@soko04.front.sepia.ceph.com success: true 2026-02-26T03:58:18.282 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-26T03:58:18.395 INFO:teuthology.run:pass