2026-02-25T08:01:19.800 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-25T08:01:19.818 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T08:01:19.896 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69032 branch: wip-logstoremanager description: crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{default} thrashosds-health workloads/snaps-few-objects-localized} email: null first_in_suite: false flavor: debug job_id: '69032' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-logstoremanager ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 osd pool default crimson: true paxos service trim min: 10 osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd max markdown count: 1000 osd objectstore: seastore osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 osd target transaction size: 5 seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - Degraded data redundancy - is down - osds down - pg .*? is .*?degraded.*?, acting - pg .*? is stuck - pg degraded - PG_BACKFILL_FULL - Low space hindering backfill .*? backfill_toofull - OSD_ROOT_DOWN - pgs degraded - pgs undersized - is active.*backfill_toofull.* sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: debug sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 thrashosds: min_in: 4 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: c32469671c6b0222f786dd36c34eb702f780a11d owner: scheduled_matan@soko04.front.sepia.ceph.com priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 869 sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: c32469671c6b0222f786dd36c34eb702f780a11d targets: trial006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBY8KPfLJtrb5ih0DtRFYMLj6uAIodB5iOSbqQAMsycvZ7aFsIfsOyn49oQfJzjkN4Wu7dyyiXnf41krlwJyqIk= trial158.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCz43pSNwRLgrgH+0Ke9zmdsvWRMOFwzOEozp2jrQafGic7WXD6ygD3194phBL/Wshr/5nhbtjhnbTLtSTWy62Y= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: ceph_objectstore_tool: false chance_inject_pause_short: 0 chance_reset_purged_snaps_last: 0 chance_thrash_cluster_full: 0 chance_thrash_pg_upmap: 0 chance_trim_stale_osdmaps: 0 dump_ops_enable: false min_in: 3 noscrub_toggle_delay: 0 reweight_osd: 0 sighup_delay: 0 thrash_primary_affinity: false timeout: 2400 - rados: clients: - client.0 localize_reads: true objects: 50 op_weights: copy_from: 0 delete: 50 read: 100 rollback: 0 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-25_07:55:35 tube: trial user: matan verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-25T08:01:19.896 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa; will attempt to use it 2026-02-25T08:01:19.896 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks 2026-02-25T08:01:19.896 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-25T08:01:19.898 INFO:teuthology.task.internal:Checking packages... 2026-02-25T08:01:19.898 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '6aeb6975bc9a4790a399c9cc70dba9635d2a7819' 2026-02-25T08:01:19.898 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-25T08:01:19.898 INFO:teuthology.packaging:ref: None 2026-02-25T08:01:19.898 INFO:teuthology.packaging:tag: None 2026-02-25T08:01:19.898 INFO:teuthology.packaging:branch: wip-logstoremanager 2026-02-25T08:01:19.898 INFO:teuthology.packaging:sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:01:19.898 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-logstoremanager 2026-02-25T08:01:20.015 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5338.g6aeb6975 2026-02-25T08:01:20.016 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-25T08:01:20.028 INFO:teuthology.task.internal:no buildpackages task found 2026-02-25T08:01:20.028 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-25T08:01:20.041 INFO:teuthology.task.internal:Saving configuration 2026-02-25T08:01:20.055 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-25T08:01:20.067 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-25T08:01:20.141 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial006.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69032', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 07:57:27.642076', 'locked_by': 'scheduled_matan@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBY8KPfLJtrb5ih0DtRFYMLj6uAIodB5iOSbqQAMsycvZ7aFsIfsOyn49oQfJzjkN4Wu7dyyiXnf41krlwJyqIk='} 2026-02-25T08:01:20.195 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial158.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69032', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 07:57:27.642693', 'locked_by': 'scheduled_matan@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCz43pSNwRLgrgH+0Ke9zmdsvWRMOFwzOEozp2jrQafGic7WXD6ygD3194phBL/Wshr/5nhbtjhnbTLtSTWy62Y='} 2026-02-25T08:01:20.195 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-25T08:01:20.206 INFO:teuthology.task.internal:roles: ubuntu@trial006.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2026-02-25T08:01:20.206 INFO:teuthology.task.internal:roles: ubuntu@trial158.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2026-02-25T08:01:20.206 INFO:teuthology.run_tasks:Running task console_log... 2026-02-25T08:01:20.335 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7e1ac8daf1a0>, signals=[15]) 2026-02-25T08:01:20.335 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-25T08:01:20.353 INFO:teuthology.task.internal:Opening connections... 2026-02-25T08:01:20.353 DEBUG:teuthology.task.internal:connecting to ubuntu@trial006.front.sepia.ceph.com 2026-02-25T08:01:20.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:01:20.428 DEBUG:teuthology.task.internal:connecting to ubuntu@trial158.front.sepia.ceph.com 2026-02-25T08:01:20.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:01:20.489 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-25T08:01:20.500 DEBUG:teuthology.orchestra.run.trial006:> uname -m 2026-02-25T08:01:20.516 INFO:teuthology.orchestra.run.trial006.stdout:x86_64 2026-02-25T08:01:20.517 DEBUG:teuthology.orchestra.run.trial006:> cat /etc/os-release 2026-02-25T08:01:20.571 INFO:teuthology.orchestra.run.trial006.stdout:NAME="CentOS Stream" 2026-02-25T08:01:20.571 INFO:teuthology.orchestra.run.trial006.stdout:VERSION="9" 2026-02-25T08:01:20.571 INFO:teuthology.orchestra.run.trial006.stdout:ID="centos" 2026-02-25T08:01:20.572 INFO:teuthology.orchestra.run.trial006.stdout:ID_LIKE="rhel fedora" 2026-02-25T08:01:20.572 INFO:teuthology.orchestra.run.trial006.stdout:VERSION_ID="9" 2026-02-25T08:01:20.572 INFO:teuthology.orchestra.run.trial006.stdout:PLATFORM_ID="platform:el9" 2026-02-25T08:01:20.572 INFO:teuthology.orchestra.run.trial006.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T08:01:20.572 INFO:teuthology.orchestra.run.trial006.stdout:ANSI_COLOR="0;31" 2026-02-25T08:01:20.572 INFO:teuthology.orchestra.run.trial006.stdout:LOGO="fedora-logo-icon" 2026-02-25T08:01:20.572 INFO:teuthology.orchestra.run.trial006.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T08:01:20.572 INFO:teuthology.orchestra.run.trial006.stdout:HOME_URL="https://centos.org/" 2026-02-25T08:01:20.572 INFO:teuthology.orchestra.run.trial006.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T08:01:20.572 INFO:teuthology.orchestra.run.trial006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T08:01:20.572 INFO:teuthology.orchestra.run.trial006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T08:01:20.572 INFO:teuthology.lock.ops:Updating trial006.front.sepia.ceph.com on lock server 2026-02-25T08:01:20.631 DEBUG:teuthology.orchestra.run.trial158:> uname -m 2026-02-25T08:01:20.646 INFO:teuthology.orchestra.run.trial158.stdout:x86_64 2026-02-25T08:01:20.646 DEBUG:teuthology.orchestra.run.trial158:> cat /etc/os-release 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:NAME="CentOS Stream" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:VERSION="9" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:ID="centos" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:ID_LIKE="rhel fedora" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:VERSION_ID="9" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:PLATFORM_ID="platform:el9" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:ANSI_COLOR="0;31" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:LOGO="fedora-logo-icon" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:HOME_URL="https://centos.org/" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T08:01:20.702 INFO:teuthology.orchestra.run.trial158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T08:01:20.702 INFO:teuthology.lock.ops:Updating trial158.front.sepia.ceph.com on lock server 2026-02-25T08:01:20.786 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-25T08:01:20.799 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-25T08:01:20.812 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-25T08:01:20.812 DEBUG:teuthology.orchestra.run.trial006:> test '!' -e /home/ubuntu/cephtest 2026-02-25T08:01:20.814 DEBUG:teuthology.orchestra.run.trial158:> test '!' -e /home/ubuntu/cephtest 2026-02-25T08:01:20.829 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-25T08:01:20.839 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-25T08:01:20.839 DEBUG:teuthology.orchestra.run.trial006:> test -z $(ls -A /var/lib/ceph) 2026-02-25T08:01:20.869 DEBUG:teuthology.orchestra.run.trial158:> test -z $(ls -A /var/lib/ceph) 2026-02-25T08:01:20.885 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-25T08:01:21.009 INFO:teuthology.run_tasks:Running task kernel... 2026-02-25T08:01:21.026 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-25T08:01:21.027 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-25T08:01:21.027 DEBUG:teuthology.orchestra.run.trial006:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T08:01:21.027 DEBUG:teuthology.orchestra.run.trial158:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T08:01:21.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:01:21.041 DEBUG:teuthology.orchestra.run.trial006:> uname -r 2026-02-25T08:01:21.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:01:21.042 DEBUG:teuthology.orchestra.run.trial158:> uname -r 2026-02-25T08:01:21.096 INFO:teuthology.orchestra.run.trial158.stdout:5.14.0-681.el9.x86_64 2026-02-25T08:01:21.096 INFO:teuthology.task.kernel:Running kernel on trial158: 5.14.0-681.el9.x86_64 2026-02-25T08:01:21.097 DEBUG:teuthology.orchestra.run.trial158:> sudo yum install -y kernel 2026-02-25T08:01:21.097 INFO:teuthology.orchestra.run.trial006.stdout:5.14.0-681.el9.x86_64 2026-02-25T08:01:21.097 INFO:teuthology.task.kernel:Running kernel on trial006: 5.14.0-681.el9.x86_64 2026-02-25T08:01:21.097 DEBUG:teuthology.orchestra.run.trial006:> sudo yum install -y kernel 2026-02-25T08:01:22.396 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.9 MB 00:00 2026-02-25T08:01:22.440 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.9 MB 00:00 2026-02-25T08:01:23.301 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - AppStream 72 MB/s | 27 MB 00:00 2026-02-25T08:01:25.878 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 27 MB 00:02 2026-02-25T08:01:29.272 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 8.0 MB 00:00 2026-02-25T08:01:30.110 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - Extras packages 235 kB/s | 20 kB 00:00 2026-02-25T08:01:30.827 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 8.0 MB 00:05 2026-02-25T08:01:31.673 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - Extras packages 234 kB/s | 20 kB 00:00 2026-02-25T08:01:32.526 INFO:teuthology.orchestra.run.trial158.stdout:Extra Packages for Enterprise Linux 8.6 MB/s | 20 MB 00:02 2026-02-25T08:01:32.728 INFO:teuthology.orchestra.run.trial006.stdout:Extra Packages for Enterprise Linux 21 MB/s | 20 MB 00:00 2026-02-25T08:01:35.740 INFO:teuthology.orchestra.run.trial158.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-25T08:01:36.036 INFO:teuthology.orchestra.run.trial006.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-25T08:01:36.746 INFO:teuthology.orchestra.run.trial158.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-25T08:01:36.746 INFO:teuthology.orchestra.run.trial158.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-25T08:01:36.766 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:01:36.767 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:01:36.768 INFO:teuthology.orchestra.run.trial158.stdout: Package Architecture Version Repository Size 2026-02-25T08:01:36.768 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:01:36.768 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:01:36.768 INFO:teuthology.orchestra.run.trial158.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-25T08:01:36.768 INFO:teuthology.orchestra.run.trial158.stdout:Installing dependencies: 2026-02-25T08:01:36.768 INFO:teuthology.orchestra.run.trial158.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-25T08:01:36.768 INFO:teuthology.orchestra.run.trial158.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-25T08:01:36.768 INFO:teuthology.orchestra.run.trial158.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-25T08:01:36.768 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:01:36.769 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:01:36.769 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:01:36.769 INFO:teuthology.orchestra.run.trial158.stdout:Install 4 Packages 2026-02-25T08:01:36.769 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:01:36.769 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 89 M 2026-02-25T08:01:36.769 INFO:teuthology.orchestra.run.trial158.stdout:Installed size: 135 M 2026-02-25T08:01:36.769 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:01:37.046 INFO:teuthology.orchestra.run.trial006.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-25T08:01:37.046 INFO:teuthology.orchestra.run.trial006.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-25T08:01:37.055 INFO:teuthology.orchestra.run.trial158.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 4.4 MB/s | 901 kB 00:00 2026-02-25T08:01:37.066 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:01:37.067 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:01:37.067 INFO:teuthology.orchestra.run.trial006.stdout: Package Architecture Version Repository Size 2026-02-25T08:01:37.067 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:01:37.067 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:01:37.067 INFO:teuthology.orchestra.run.trial006.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-25T08:01:37.067 INFO:teuthology.orchestra.run.trial006.stdout:Installing dependencies: 2026-02-25T08:01:37.067 INFO:teuthology.orchestra.run.trial006.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-25T08:01:37.067 INFO:teuthology.orchestra.run.trial006.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-25T08:01:37.067 INFO:teuthology.orchestra.run.trial006.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-25T08:01:37.067 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:01:37.068 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:01:37.068 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:01:37.068 INFO:teuthology.orchestra.run.trial006.stdout:Install 4 Packages 2026-02-25T08:01:37.068 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:01:37.068 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 89 M 2026-02-25T08:01:37.068 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 135 M 2026-02-25T08:01:37.068 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:01:37.370 INFO:teuthology.orchestra.run.trial158.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 33 MB/s | 17 MB 00:00 2026-02-25T08:01:37.396 INFO:teuthology.orchestra.run.trial006.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 13 MB/s | 901 kB 00:00 2026-02-25T08:01:37.480 INFO:teuthology.orchestra.run.trial006.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 114 MB/s | 17 MB 00:00 2026-02-25T08:01:37.567 INFO:teuthology.orchestra.run.trial006.stdout:(3/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 167 MB/s | 40 MB 00:00 2026-02-25T08:01:37.643 INFO:teuthology.orchestra.run.trial006.stdout:(4/4): kernel-modules-core-5.14.0-686.el9.x86_6 126 MB/s | 31 MB 00:00 2026-02-25T08:01:37.645 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:01:37.645 INFO:teuthology.orchestra.run.trial006.stdout:Total 154 MB/s | 89 MB 00:00 2026-02-25T08:01:37.651 INFO:teuthology.orchestra.run.trial158.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 52 MB/s | 31 MB 00:00 2026-02-25T08:01:37.810 INFO:teuthology.orchestra.run.trial158.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 42 MB/s | 40 MB 00:00 2026-02-25T08:01:37.810 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:01:37.810 INFO:teuthology.orchestra.run.trial158.stdout:Total 85 MB/s | 89 MB 00:01 2026-02-25T08:01:37.840 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:01:38.000 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:01:38.043 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:01:38.043 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:01:38.201 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:01:38.201 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:01:38.426 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:01:38.426 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:01:38.584 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:01:38.584 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:01:39.540 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:01:39.693 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:01:39.829 INFO:teuthology.orchestra.run.trial006.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-25T08:01:39.851 INFO:teuthology.orchestra.run.trial006.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:01:39.982 INFO:teuthology.orchestra.run.trial158.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-25T08:01:40.005 INFO:teuthology.orchestra.run.trial158.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:01:42.414 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:01:42.448 INFO:teuthology.orchestra.run.trial006.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:01:42.579 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:01:42.612 INFO:teuthology.orchestra.run.trial158.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:01:45.653 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:01:45.659 INFO:teuthology.orchestra.run.trial158.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:01:45.739 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:01:45.746 INFO:teuthology.orchestra.run.trial006.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:01:48.708 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:01:48.803 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:02:06.258 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:02:06.279 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:02:06.285 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:02:06.306 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:02:06.465 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:02:06.465 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-25T08:02:06.465 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:02:06.465 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:02:06.491 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:02:06.491 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-25T08:02:06.491 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:02:06.491 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:02:06.523 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:02:06.523 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:02:06.523 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:02:06.524 INFO:teuthology.orchestra.run.trial006.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.524 INFO:teuthology.orchestra.run.trial006.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.524 INFO:teuthology.orchestra.run.trial006.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.524 INFO:teuthology.orchestra.run.trial006.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.524 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:02:06.524 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:02:06.548 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:02:06.548 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:02:06.548 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:02:06.549 INFO:teuthology.orchestra.run.trial158.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.549 INFO:teuthology.orchestra.run.trial158.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.549 INFO:teuthology.orchestra.run.trial158.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.549 INFO:teuthology.orchestra.run.trial158.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.549 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:02:06.549 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:02:06.584 DEBUG:teuthology.orchestra.run.trial006:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T08:02:06.608 DEBUG:teuthology.orchestra.run.trial158:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T08:02:06.613 INFO:teuthology.orchestra.run.trial006.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.614 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T08:02:06.614 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-25T08:02:06.615 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-25T08:02:06.615 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-25T08:02:06.615 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-25T08:02:06.615 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial006.front.sepia.ceph.com, path=None, version=distro) 2026-02-25T08:02:06.615 DEBUG:teuthology.orchestra.run.trial006:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T08:02:06.638 INFO:teuthology.orchestra.run.trial158.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.639 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T08:02:06.639 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-25T08:02:06.639 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-25T08:02:06.639 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-25T08:02:06.639 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2026-02-25T08:02:06.639 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial158.front.sepia.ceph.com, path=None, version=distro) 2026-02-25T08:02:06.639 DEBUG:teuthology.orchestra.run.trial158:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T08:02:06.681 INFO:teuthology.orchestra.run.trial006.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.683 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T08:02:06.683 DEBUG:teuthology.orchestra.run.trial006:> sudo rpm -qi grub2-tools 2026-02-25T08:02:06.704 INFO:teuthology.orchestra.run.trial158.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.705 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T08:02:06.706 DEBUG:teuthology.orchestra.run.trial158:> sudo rpm -qi grub2-tools 2026-02-25T08:02:06.751 INFO:teuthology.orchestra.run.trial006.stdout:Name : grub2-tools 2026-02-25T08:02:06.751 INFO:teuthology.orchestra.run.trial006.stdout:Epoch : 1 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Version : 2.06 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Release : 120.el9 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Architecture: x86_64 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Group : Unspecified 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Size : 9108611 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:License : GPLv3+ 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Packager : builder@centos.org 2026-02-25T08:02:06.752 INFO:teuthology.orchestra.run.trial006.stdout:Vendor : CentOS 2026-02-25T08:02:06.753 INFO:teuthology.orchestra.run.trial006.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T08:02:06.753 INFO:teuthology.orchestra.run.trial006.stdout:Summary : Support tools for GRUB. 2026-02-25T08:02:06.753 INFO:teuthology.orchestra.run.trial006.stdout:Description : 2026-02-25T08:02:06.753 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:02:06.753 INFO:teuthology.orchestra.run.trial006.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T08:02:06.753 INFO:teuthology.orchestra.run.trial006.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T08:02:06.753 INFO:teuthology.orchestra.run.trial006.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T08:02:06.753 INFO:teuthology.orchestra.run.trial006.stdout:hardware devices. 2026-02-25T08:02:06.753 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:02:06.753 INFO:teuthology.orchestra.run.trial006.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T08:02:06.754 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T08:02:06.755 INFO:teuthology.task.kernel:Updating grub on trial006 to boot 5.14.0-686.el9.x86_64 2026-02-25T08:02:06.755 DEBUG:teuthology.orchestra.run.trial006:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T08:02:06.774 INFO:teuthology.orchestra.run.trial158.stdout:Name : grub2-tools 2026-02-25T08:02:06.774 INFO:teuthology.orchestra.run.trial158.stdout:Epoch : 1 2026-02-25T08:02:06.774 INFO:teuthology.orchestra.run.trial158.stdout:Version : 2.06 2026-02-25T08:02:06.774 INFO:teuthology.orchestra.run.trial158.stdout:Release : 120.el9 2026-02-25T08:02:06.774 INFO:teuthology.orchestra.run.trial158.stdout:Architecture: x86_64 2026-02-25T08:02:06.774 INFO:teuthology.orchestra.run.trial158.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T08:02:06.774 INFO:teuthology.orchestra.run.trial158.stdout:Group : Unspecified 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout:Size : 9108611 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout:License : GPLv3+ 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout:Packager : builder@centos.org 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout:Vendor : CentOS 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout:Summary : Support tools for GRUB. 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout:Description : 2026-02-25T08:02:06.775 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:02:06.776 INFO:teuthology.orchestra.run.trial158.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T08:02:06.776 INFO:teuthology.orchestra.run.trial158.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T08:02:06.776 INFO:teuthology.orchestra.run.trial158.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T08:02:06.776 INFO:teuthology.orchestra.run.trial158.stdout:hardware devices. 2026-02-25T08:02:06.776 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:02:06.776 INFO:teuthology.orchestra.run.trial158.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T08:02:06.777 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T08:02:06.777 INFO:teuthology.task.kernel:Updating grub on trial158 to boot 5.14.0-686.el9.x86_64 2026-02-25T08:02:06.777 DEBUG:teuthology.orchestra.run.trial158:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T08:02:06.818 INFO:teuthology.orchestra.run.trial006.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T08:02:06.819 DEBUG:teuthology.orchestra.run.trial006:> sudo command -v grubby && echo yes || echo no 2026-02-25T08:02:06.840 INFO:teuthology.orchestra.run.trial158.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T08:02:06.841 DEBUG:teuthology.orchestra.run.trial158:> sudo command -v grubby && echo yes || echo no 2026-02-25T08:02:06.882 INFO:teuthology.orchestra.run.trial006.stdout:/sbin/grubby 2026-02-25T08:02:06.884 INFO:teuthology.orchestra.run.trial006.stdout:yes 2026-02-25T08:02:06.885 DEBUG:teuthology.orchestra.run.trial006:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-25T08:02:06.904 INFO:teuthology.orchestra.run.trial158.stdout:/sbin/grubby 2026-02-25T08:02:06.904 INFO:teuthology.orchestra.run.trial158.stdout:yes 2026-02-25T08:02:06.905 DEBUG:teuthology.orchestra.run.trial158:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-25T08:02:06.948 INFO:teuthology.orchestra.run.trial006.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-25T08:02:06.950 DEBUG:teuthology.orchestra.run.trial006:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-25T08:02:06.970 INFO:teuthology.orchestra.run.trial158.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-25T08:02:06.972 DEBUG:teuthology.orchestra.run.trial158:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-25T08:02:07.023 DEBUG:teuthology.orchestra.run.trial006:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T08:02:07.044 DEBUG:teuthology.orchestra.run.trial158:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T08:02:07.230 INFO:teuthology.orchestra.run.trial006.stderr:Generating grub configuration file ... 2026-02-25T08:02:07.257 INFO:teuthology.orchestra.run.trial158.stderr:Generating grub configuration file ... 2026-02-25T08:02:07.298 INFO:teuthology.orchestra.run.trial006.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-25T08:02:07.305 INFO:teuthology.orchestra.run.trial006.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-25T08:02:07.325 INFO:teuthology.orchestra.run.trial158.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-25T08:02:07.332 INFO:teuthology.orchestra.run.trial158.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-25T08:02:07.560 INFO:teuthology.orchestra.run.trial006.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T08:02:07.567 INFO:teuthology.orchestra.run.trial006.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T08:02:07.585 INFO:teuthology.orchestra.run.trial158.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T08:02:07.591 INFO:teuthology.orchestra.run.trial158.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T08:02:07.601 INFO:teuthology.orchestra.run.trial006.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T08:02:07.607 INFO:teuthology.orchestra.run.trial006.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T08:02:07.627 INFO:teuthology.orchestra.run.trial158.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T08:02:07.633 INFO:teuthology.orchestra.run.trial158.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T08:02:07.638 INFO:teuthology.orchestra.run.trial006.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T08:02:07.645 INFO:teuthology.orchestra.run.trial006.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T08:02:07.664 INFO:teuthology.orchestra.run.trial158.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T08:02:07.669 INFO:teuthology.orchestra.run.trial006.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T08:02:07.671 INFO:teuthology.orchestra.run.trial158.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T08:02:07.676 INFO:teuthology.orchestra.run.trial006.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T08:02:07.695 INFO:teuthology.orchestra.run.trial158.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T08:02:07.701 INFO:teuthology.orchestra.run.trial158.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T08:02:07.764 INFO:teuthology.orchestra.run.trial006.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T08:02:07.773 INFO:teuthology.orchestra.run.trial006.stderr:done 2026-02-25T08:02:07.775 DEBUG:teuthology.orchestra.run.trial006:> test -d /sys/firmware/efi 2026-02-25T08:02:07.788 INFO:teuthology.orchestra.run.trial158.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T08:02:07.791 DEBUG:teuthology.orchestra.run.trial006:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T08:02:07.796 INFO:teuthology.orchestra.run.trial158.stderr:done 2026-02-25T08:02:07.799 DEBUG:teuthology.orchestra.run.trial158:> test -d /sys/firmware/efi 2026-02-25T08:02:07.815 DEBUG:teuthology.orchestra.run.trial158:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T08:02:07.856 INFO:teuthology.orchestra.run.trial006.stdout:/boot/efi/EFI/BOOT 2026-02-25T08:02:07.856 INFO:teuthology.orchestra.run.trial006.stdout:/boot/efi/EFI/centos 2026-02-25T08:02:07.857 DEBUG:teuthology.orchestra.run.trial006:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T08:02:07.882 INFO:teuthology.orchestra.run.trial158.stdout:/boot/efi/EFI/BOOT 2026-02-25T08:02:07.882 INFO:teuthology.orchestra.run.trial158.stdout:/boot/efi/EFI/centos 2026-02-25T08:02:07.884 DEBUG:teuthology.orchestra.run.trial158:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T08:02:07.923 INFO:teuthology.orchestra.run.trial006.stdout:no 2026-02-25T08:02:07.923 DEBUG:teuthology.orchestra.run.trial006:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T08:02:07.948 INFO:teuthology.orchestra.run.trial158.stdout:no 2026-02-25T08:02:07.949 DEBUG:teuthology.orchestra.run.trial158:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T08:02:07.988 INFO:teuthology.orchestra.run.trial006.stdout:yes 2026-02-25T08:02:07.988 DEBUG:teuthology.orchestra.run.trial006:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T08:02:08.013 INFO:teuthology.orchestra.run.trial158.stdout:yes 2026-02-25T08:02:08.013 DEBUG:teuthology.orchestra.run.trial158:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T08:02:08.054 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial006 2026-02-25T08:02:08.055 DEBUG:teuthology.orchestra.run.trial006:> sudo bash -c ' 2026-02-25T08:02:08.055 DEBUG:teuthology.orchestra.run.trial006:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T08:02:08.055 DEBUG:teuthology.orchestra.run.trial006:> ' 2026-02-25T08:02:08.079 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial158 2026-02-25T08:02:08.080 DEBUG:teuthology.orchestra.run.trial158:> sudo bash -c ' 2026-02-25T08:02:08.080 DEBUG:teuthology.orchestra.run.trial158:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T08:02:08.080 DEBUG:teuthology.orchestra.run.trial158:> ' 2026-02-25T08:02:38.097 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-25T08:02:38.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial006.front.sepia.ceph.com' 2026-02-25T08:02:38.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:02:38.122 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2026-02-25T08:02:38.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial158.front.sepia.ceph.com' 2026-02-25T08:02:38.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:02:52.352 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.6 2026-02-25T08:02:52.416 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.158 2026-02-25T08:03:01.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial006.front.sepia.ceph.com' 2026-02-25T08:03:01.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:03:01.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial158.front.sepia.ceph.com' 2026-02-25T08:03:01.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:03:37.621 DEBUG:teuthology.orchestra.run.trial158:> true 2026-02-25T08:03:37.643 DEBUG:teuthology.orchestra.run.trial006:> true 2026-02-25T08:03:37.749 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial158.front.sepia.ceph.com' 2026-02-25T08:03:37.749 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-686.el9.x86_64"... 2026-02-25T08:03:37.749 DEBUG:teuthology.orchestra.run.trial158:> uname -r 2026-02-25T08:03:37.803 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial006.front.sepia.ceph.com' 2026-02-25T08:03:37.804 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-686.el9.x86_64"... 2026-02-25T08:03:37.804 DEBUG:teuthology.orchestra.run.trial006:> uname -r 2026-02-25T08:03:37.804 INFO:teuthology.orchestra.run.trial158.stdout:5.14.0-686.el9.x86_64 2026-02-25T08:03:37.804 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-25T08:03:37.804 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T08:03:37.805 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T08:03:37.860 INFO:teuthology.orchestra.run.trial006.stdout:5.14.0-686.el9.x86_64 2026-02-25T08:03:37.861 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-25T08:03:37.861 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T08:03:37.861 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T08:03:38.805 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2026-02-25T08:03:38.806 DEBUG:teuthology.orchestra.run.trial158:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-25T08:03:38.836 INFO:teuthology.orchestra.run.trial158.stdout:ttyS1 2026-02-25T08:03:38.862 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-25T08:03:38.863 DEBUG:teuthology.orchestra.run.trial006:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-25T08:03:38.868 DEBUG:teuthology.parallel:result is None 2026-02-25T08:03:38.892 INFO:teuthology.orchestra.run.trial006.stdout:ttyS1 2026-02-25T08:03:38.913 DEBUG:teuthology.parallel:result is None 2026-02-25T08:03:38.914 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-25T08:03:38.926 INFO:teuthology.task.internal:Creating test directory... 2026-02-25T08:03:38.927 DEBUG:teuthology.orchestra.run.trial006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T08:03:38.956 DEBUG:teuthology.orchestra.run.trial158:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T08:03:38.974 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-25T08:03:38.985 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-25T08:03:38.999 INFO:teuthology.task.internal:Creating archive directory... 2026-02-25T08:03:38.999 DEBUG:teuthology.orchestra.run.trial006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T08:03:39.012 DEBUG:teuthology.orchestra.run.trial158:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T08:03:39.036 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-25T08:03:39.048 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-25T08:03:39.049 DEBUG:teuthology.orchestra.run.trial006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T08:03:39.069 DEBUG:teuthology.orchestra.run.trial158:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T08:03:39.091 INFO:teuthology.orchestra.run.trial006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T08:03:39.093 INFO:teuthology.orchestra.run.trial158.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T08:03:39.100 INFO:teuthology.orchestra.run.trial006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T08:03:39.102 INFO:teuthology.orchestra.run.trial158.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T08:03:39.103 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-25T08:03:39.116 INFO:teuthology.task.internal:Configuring sudo... 2026-02-25T08:03:39.116 DEBUG:teuthology.orchestra.run.trial006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T08:03:39.144 DEBUG:teuthology.orchestra.run.trial158:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T08:03:39.167 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-25T08:03:39.182 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-25T08:03:39.182 DEBUG:teuthology.orchestra.run.trial006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T08:03:39.208 DEBUG:teuthology.orchestra.run.trial158:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T08:03:39.224 DEBUG:teuthology.orchestra.run.trial006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:03:39.382 DEBUG:teuthology.orchestra.run.trial006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:03:39.407 DEBUG:teuthology.orchestra.run.trial006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:03:39.560 DEBUG:teuthology.orchestra.run.trial006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:03:39.582 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:03:39.582 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T08:03:39.646 DEBUG:teuthology.orchestra.run.trial158:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:03:39.743 DEBUG:teuthology.orchestra.run.trial158:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:03:39.769 DEBUG:teuthology.orchestra.run.trial158:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:03:39.916 DEBUG:teuthology.orchestra.run.trial158:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:03:39.941 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:03:39.941 DEBUG:teuthology.orchestra.run.trial158:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T08:03:40.003 DEBUG:teuthology.orchestra.run.trial006:> sudo service rsyslog restart 2026-02-25T08:03:40.005 DEBUG:teuthology.orchestra.run.trial158:> sudo service rsyslog restart 2026-02-25T08:03:40.032 INFO:teuthology.orchestra.run.trial006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T08:03:40.070 INFO:teuthology.orchestra.run.trial158.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T08:03:40.593 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-25T08:03:40.604 INFO:teuthology.task.internal:Starting timer... 2026-02-25T08:03:40.604 INFO:teuthology.run_tasks:Running task pcp... 2026-02-25T08:03:40.616 INFO:teuthology.run_tasks:Running task selinux... 2026-02-25T08:03:40.634 DEBUG:teuthology.orchestra.run.trial006:> sudo service auditd rotate 2026-02-25T08:03:40.665 INFO:teuthology.orchestra.run.trial006.stdout:Rotating logs: 2026-02-25T08:03:40.667 DEBUG:teuthology.orchestra.run.trial158:> sudo service auditd rotate 2026-02-25T08:03:40.696 INFO:teuthology.orchestra.run.trial158.stdout:Rotating logs: 2026-02-25T08:03:40.698 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-25T08:03:40.698 DEBUG:teuthology.orchestra.run.trial006:> /usr/sbin/getenforce 2026-02-25T08:03:40.723 INFO:teuthology.orchestra.run.trial006.stdout:Permissive 2026-02-25T08:03:40.724 DEBUG:teuthology.orchestra.run.trial158:> /usr/sbin/getenforce 2026-02-25T08:03:40.754 INFO:teuthology.orchestra.run.trial158.stdout:Permissive 2026-02-25T08:03:40.754 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial006.front.sepia.ceph.com': 'permissive', 'ubuntu@trial158.front.sepia.ceph.com': 'permissive'} 2026-02-25T08:03:40.755 DEBUG:teuthology.orchestra.run.trial006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T08:03:40.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:03:40.787 DEBUG:teuthology.orchestra.run.trial158:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T08:03:40.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:03:40.819 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-25T08:03:40.819 DEBUG:teuthology.orchestra.run.trial006:> sudo /usr/sbin/setenforce permissive 2026-02-25T08:03:40.849 DEBUG:teuthology.orchestra.run.trial158:> sudo /usr/sbin/setenforce permissive 2026-02-25T08:03:40.884 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-25T08:03:40.899 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-25T08:03:40.906 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-25T08:03:40.907 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial006.front.sepia.ceph.com,trial158.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-25T08:06:36.340 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial006.front.sepia.ceph.com'), Remote(name='ubuntu@trial158.front.sepia.ceph.com')] 2026-02-25T08:06:36.340 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial006.front.sepia.ceph.com' 2026-02-25T08:06:36.341 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:06:36.427 DEBUG:teuthology.orchestra.run.trial006:> true 2026-02-25T08:06:36.498 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial006.front.sepia.ceph.com' 2026-02-25T08:06:36.498 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial158.front.sepia.ceph.com' 2026-02-25T08:06:36.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:06:36.562 DEBUG:teuthology.orchestra.run.trial158:> true 2026-02-25T08:06:36.636 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial158.front.sepia.ceph.com' 2026-02-25T08:06:36.636 INFO:teuthology.run_tasks:Running task clock... 2026-02-25T08:06:36.649 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-25T08:06:36.649 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T08:06:36.649 DEBUG:teuthology.orchestra.run.trial006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T08:06:36.651 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T08:06:36.651 DEBUG:teuthology.orchestra.run.trial158:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T08:06:36.677 INFO:teuthology.orchestra.run.trial006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T08:06:36.689 INFO:teuthology.orchestra.run.trial006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T08:06:36.703 INFO:teuthology.orchestra.run.trial158.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T08:06:36.709 INFO:teuthology.orchestra.run.trial006.stderr:sudo: ntpd: command not found 2026-02-25T08:06:36.715 INFO:teuthology.orchestra.run.trial158.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T08:06:36.718 INFO:teuthology.orchestra.run.trial006.stdout:506 Cannot talk to daemon 2026-02-25T08:06:36.729 INFO:teuthology.orchestra.run.trial006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T08:06:36.736 INFO:teuthology.orchestra.run.trial158.stderr:sudo: ntpd: command not found 2026-02-25T08:06:36.741 INFO:teuthology.orchestra.run.trial006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T08:06:36.746 INFO:teuthology.orchestra.run.trial158.stdout:506 Cannot talk to daemon 2026-02-25T08:06:36.758 INFO:teuthology.orchestra.run.trial158.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T08:06:36.770 INFO:teuthology.orchestra.run.trial158.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T08:06:36.791 INFO:teuthology.orchestra.run.trial006.stderr:bash: line 1: ntpq: command not found 2026-02-25T08:06:36.815 INFO:teuthology.orchestra.run.trial006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T08:06:36.815 INFO:teuthology.orchestra.run.trial006.stdout:=============================================================================== 2026-02-25T08:06:36.815 INFO:teuthology.orchestra.run.trial006.stdout:^? ntpool0.603.newcontinuum> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T08:06:36.815 INFO:teuthology.orchestra.run.trial006.stdout:^? time.cloudflare.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T08:06:36.815 INFO:teuthology.orchestra.run.trial006.stdout:^? 96-19-94-82.cpe.sparklig> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T08:06:36.815 INFO:teuthology.orchestra.run.trial006.stdout:^? 170-187-142-180.ip.linod> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T08:06:36.825 INFO:teuthology.orchestra.run.trial158.stderr:bash: line 1: ntpq: command not found 2026-02-25T08:06:36.987 INFO:teuthology.orchestra.run.trial158.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T08:06:36.987 INFO:teuthology.orchestra.run.trial158.stdout:=============================================================================== 2026-02-25T08:06:36.987 INFO:teuthology.orchestra.run.trial158.stdout:^? time.lmtlabs.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T08:06:36.987 INFO:teuthology.orchestra.run.trial158.stdout:^? 73.185.182.209 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T08:06:36.987 INFO:teuthology.orchestra.run.trial158.stdout:^? usmoi.meezy.link 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T08:06:36.987 INFO:teuthology.orchestra.run.trial158.stdout:^? 173.249.203.227 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T08:06:36.988 INFO:teuthology.run_tasks:Running task install... 2026-02-25T08:06:36.999 DEBUG:teuthology.task.install:project ceph 2026-02-25T08:06:36.999 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}} 2026-02-25T08:06:36.999 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'} 2026-02-25T08:06:36.999 INFO:teuthology.task.install:Using flavor: debug 2026-02-25T08:06:37.002 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-25T08:06:37.002 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-25T08:06:37.002 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'tag': None, 'wait_for_package': False} 2026-02-25T08:06:37.002 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:06:37.003 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'tag': None, 'wait_for_package': False} 2026-02-25T08:06:37.004 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:06:37.165 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/ 2026-02-25T08:06:37.165 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5338.g6aeb6975 2026-02-25T08:06:37.173 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/ 2026-02-25T08:06:37.173 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5338.g6aeb6975 2026-02-25T08:06:37.482 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T08:06:37.483 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:06:37.483 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T08:06:37.509 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-25T08:06:37.509 DEBUG:teuthology.orchestra.run.trial006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T08:06:37.519 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T08:06:37.520 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:06:37.520 DEBUG:teuthology.orchestra.run.trial158:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T08:06:37.545 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-25T08:06:37.546 DEBUG:teuthology.orchestra.run.trial158:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T08:06:37.573 DEBUG:teuthology.orchestra.run.trial006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T08:06:37.610 DEBUG:teuthology.orchestra.run.trial158:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T08:06:37.646 DEBUG:teuthology.orchestra.run.trial006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T08:06:37.683 DEBUG:teuthology.orchestra.run.trial158:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T08:06:37.710 INFO:teuthology.orchestra.run.trial006.stdout:check_obsoletes = 1 2026-02-25T08:06:37.711 DEBUG:teuthology.orchestra.run.trial006:> sudo yum clean all 2026-02-25T08:06:37.747 INFO:teuthology.orchestra.run.trial158.stdout:check_obsoletes = 1 2026-02-25T08:06:37.748 DEBUG:teuthology.orchestra.run.trial158:> sudo yum clean all 2026-02-25T08:06:37.861 INFO:teuthology.orchestra.run.trial006.stdout:41 files removed 2026-02-25T08:06:37.876 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-radosgw 2026-02-25T08:06:37.898 INFO:teuthology.orchestra.run.trial158.stdout:41 files removed 2026-02-25T08:06:37.911 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-radosgw 2026-02-25T08:06:38.683 INFO:teuthology.orchestra.run.trial158.stdout:ceph packages for x86_64 150 kB/s | 94 kB 00:00 2026-02-25T08:06:38.790 INFO:teuthology.orchestra.run.trial006.stdout:ceph packages for x86_64 123 kB/s | 94 kB 00:00 2026-02-25T08:06:39.574 INFO:teuthology.orchestra.run.trial006.stdout:ceph noarch packages 32 kB/s | 24 kB 00:00 2026-02-25T08:06:39.714 INFO:teuthology.orchestra.run.trial158.stdout:ceph noarch packages 24 kB/s | 24 kB 00:01 2026-02-25T08:06:40.084 INFO:teuthology.orchestra.run.trial006.stdout:ceph source packages 4.3 kB/s | 2.1 kB 00:00 2026-02-25T08:06:40.219 INFO:teuthology.orchestra.run.trial158.stdout:ceph source packages 4.3 kB/s | 2.1 kB 00:00 2026-02-25T08:06:41.214 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.9 MB 00:01 2026-02-25T08:06:41.674 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.9 MB 00:01 2026-02-25T08:06:44.135 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - AppStream 14 MB/s | 27 MB 00:01 2026-02-25T08:06:45.515 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 27 MB 00:03 2026-02-25T08:06:48.075 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 8.0 MB 00:01 2026-02-25T08:06:48.224 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - CRB 29 MB/s | 8.0 MB 00:00 2026-02-25T08:06:49.453 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 20 kB 00:00 2026-02-25T08:06:49.679 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 20 kB 00:00 2026-02-25T08:06:50.007 INFO:teuthology.orchestra.run.trial006.stdout:Extra Packages for Enterprise Linux 42 MB/s | 20 MB 00:00 2026-02-25T08:06:50.391 INFO:teuthology.orchestra.run.trial158.stdout:Extra Packages for Enterprise Linux 31 MB/s | 20 MB 00:00 2026-02-25T08:06:53.217 INFO:teuthology.orchestra.run.trial006.stdout:lab-extras 1.5 MB/s | 50 kB 00:00 2026-02-25T08:06:53.585 INFO:teuthology.orchestra.run.trial158.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-25T08:06:54.426 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repository Size 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: ceph-radosgw x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 11 M 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout:Upgrading: 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: librados2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 3.3 M 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: librbd1 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.7 M 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout:Installing dependencies: 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: ceph-base x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 5.1 M 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: ceph-common x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 21 M 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: ceph-selinux x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 25 k 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T08:06:54.428 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 860 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: libradosstriper1 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 263 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: librgw2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 5.6 M 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: python3-ceph-argparse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 45 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: python3-ceph-common x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 185 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: python3-cephfs x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 204 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: python3-rados x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 324 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: python3-rbd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 306 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: python3-rgw x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 100 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout:Installing weak dependencies: 2026-02-25T08:06:54.429 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs-daemon x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 37 k 2026-02-25T08:06:54.430 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 26 k 2026-02-25T08:06:54.430 INFO:teuthology.orchestra.run.trial006.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-25T08:06:54.430 INFO:teuthology.orchestra.run.trial006.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T08:06:54.430 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:06:54.430 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:06:54.430 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:06:54.430 INFO:teuthology.orchestra.run.trial006.stdout:Install 35 Packages 2026-02-25T08:06:54.430 INFO:teuthology.orchestra.run.trial006.stdout:Upgrade 2 Packages 2026-02-25T08:06:54.430 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:06:54.430 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 61 M 2026-02-25T08:06:54.430 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:06:54.800 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: Package Arch Version Repository Size 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: ceph-radosgw x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 11 M 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout:Upgrading: 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: librados2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 3.3 M 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: librbd1 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.7 M 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout:Installing dependencies: 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: ceph-base x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 5.1 M 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: ceph-common x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 21 M 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: ceph-selinux x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 25 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: libcephfs2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 860 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: libradosstriper1 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 263 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: librgw2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 5.6 M 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: python3-ceph-argparse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 45 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: python3-ceph-common x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 185 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: python3-cephfs x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 204 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: python3-rados x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 324 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: python3-rbd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 306 k 2026-02-25T08:06:54.802 INFO:teuthology.orchestra.run.trial158.stdout: python3-rgw x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 100 k 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout:Installing weak dependencies: 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout: libcephfs-daemon x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 37 k 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 26 k 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout:Install 35 Packages 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout:Upgrade 2 Packages 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 61 M 2026-02-25T08:06:54.803 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:06:57.662 INFO:teuthology.orchestra.run.trial158.stdout:(1/37): ceph-base-20.3.0-5338.g6aeb6975.el9.x86 2.0 MB/s | 5.1 MB 00:02 2026-02-25T08:06:57.717 INFO:teuthology.orchestra.run.trial158.stdout:(2/37): ceph-selinux-20.3.0-5338.g6aeb6975.el9. 462 kB/s | 25 kB 00:00 2026-02-25T08:06:57.771 INFO:teuthology.orchestra.run.trial158.stdout:(3/37): libcephfs-daemon-20.3.0-5338.g6aeb6975. 676 kB/s | 37 kB 00:00 2026-02-25T08:06:57.826 INFO:teuthology.orchestra.run.trial158.stdout:(4/37): libcephfs-proxy2-20.3.0-5338.g6aeb6975. 468 kB/s | 26 kB 00:00 2026-02-25T08:06:58.182 INFO:teuthology.orchestra.run.trial158.stdout:(5/37): libcephfs2-20.3.0-5338.g6aeb6975.el9.x8 2.4 MB/s | 860 kB 00:00 2026-02-25T08:06:58.311 INFO:teuthology.orchestra.run.trial158.stdout:(6/37): libradosstriper1-20.3.0-5338.g6aeb6975. 2.0 MB/s | 263 kB 00:00 2026-02-25T08:06:58.564 INFO:teuthology.orchestra.run.trial158.stdout:(7/37): ceph-radosgw-20.3.0-5338.g6aeb6975.el9. 3.2 MB/s | 11 MB 00:03 2026-02-25T08:06:58.633 INFO:teuthology.orchestra.run.trial158.stdout:(8/37): python3-ceph-argparse-20.3.0-5338.g6aeb 656 kB/s | 45 kB 00:00 2026-02-25T08:06:58.699 INFO:teuthology.orchestra.run.trial158.stdout:(9/37): python3-ceph-common-20.3.0-5338.g6aeb69 2.7 MB/s | 185 kB 00:00 2026-02-25T08:06:58.765 INFO:teuthology.orchestra.run.trial158.stdout:(10/37): python3-cephfs-20.3.0-5338.g6aeb6975.e 3.0 MB/s | 204 kB 00:00 2026-02-25T08:06:58.899 INFO:teuthology.orchestra.run.trial158.stdout:(11/37): python3-rados-20.3.0-5338.g6aeb6975.el 2.4 MB/s | 324 kB 00:00 2026-02-25T08:06:59.034 INFO:teuthology.orchestra.run.trial158.stdout:(12/37): python3-rbd-20.3.0-5338.g6aeb6975.el9. 2.2 MB/s | 306 kB 00:00 2026-02-25T08:06:59.104 INFO:teuthology.orchestra.run.trial158.stdout:(13/37): python3-rgw-20.3.0-5338.g6aeb6975.el9. 1.4 MB/s | 100 kB 00:00 2026-02-25T08:06:59.228 INFO:teuthology.orchestra.run.trial006.stdout:(1/37): ceph-base-20.3.0-5338.g6aeb6975.el9.x86 1.1 MB/s | 5.1 MB 00:04 2026-02-25T08:06:59.247 INFO:teuthology.orchestra.run.trial158.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 233 kB/s | 33 kB 00:00 2026-02-25T08:06:59.293 INFO:teuthology.orchestra.run.trial006.stdout:(2/37): ceph-selinux-20.3.0-5338.g6aeb6975.el9. 388 kB/s | 25 kB 00:00 2026-02-25T08:06:59.356 INFO:teuthology.orchestra.run.trial006.stdout:(3/37): libcephfs-daemon-20.3.0-5338.g6aeb6975. 583 kB/s | 37 kB 00:00 2026-02-25T08:06:59.410 INFO:teuthology.orchestra.run.trial158.stdout:(15/37): nvme-cli-2.16-1.el9.x86_64.rpm 7.1 MB/s | 1.2 MB 00:00 2026-02-25T08:06:59.424 INFO:teuthology.orchestra.run.trial006.stdout:(4/37): libcephfs-proxy2-20.3.0-5338.g6aeb6975. 379 kB/s | 26 kB 00:00 2026-02-25T08:06:59.452 INFO:teuthology.orchestra.run.trial158.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 2.2 MB/s | 93 kB 00:00 2026-02-25T08:06:59.548 INFO:teuthology.orchestra.run.trial158.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 1.1 MB/s | 104 kB 00:00 2026-02-25T08:06:59.589 INFO:teuthology.orchestra.run.trial158.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 3.9 MB/s | 164 kB 00:00 2026-02-25T08:06:59.612 INFO:teuthology.orchestra.run.trial158.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.8 MB/s | 160 kB 00:00 2026-02-25T08:06:59.633 INFO:teuthology.orchestra.run.trial158.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 2.1 MB/s | 45 kB 00:00 2026-02-25T08:06:59.676 INFO:teuthology.orchestra.run.trial158.stdout:(21/37): librdkafka-1.6.1-102.el9.x86_64.rpm 15 MB/s | 662 kB 00:00 2026-02-25T08:06:59.734 INFO:teuthology.orchestra.run.trial158.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2026-02-25T08:06:59.829 INFO:teuthology.orchestra.run.trial158.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 2.5 MB/s | 240 kB 00:00 2026-02-25T08:06:59.850 INFO:teuthology.orchestra.run.trial158.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 1.8 MB/s | 37 kB 00:00 2026-02-25T08:06:59.871 INFO:teuthology.orchestra.run.trial158.stdout:(25/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 3.1 MB/s | 66 kB 00:00 2026-02-25T08:07:00.112 INFO:teuthology.orchestra.run.trial006.stdout:(5/37): libcephfs2-20.3.0-5338.g6aeb6975.el9.x8 1.2 MB/s | 860 kB 00:00 2026-02-25T08:07:00.228 INFO:teuthology.orchestra.run.trial158.stdout:(26/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 860 kB/s | 308 kB 00:00 2026-02-25T08:07:00.305 INFO:teuthology.orchestra.run.trial006.stdout:(6/37): libradosstriper1-20.3.0-5338.g6aeb6975. 1.3 MB/s | 263 kB 00:00 2026-02-25T08:07:00.523 INFO:teuthology.orchestra.run.trial158.stdout:(27/37): libarrow-9.0.0-15.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2026-02-25T08:07:00.583 INFO:teuthology.orchestra.run.trial158.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 408 kB/s | 25 kB 00:00 2026-02-25T08:07:00.644 INFO:teuthology.orchestra.run.trial158.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 812 kB/s | 49 kB 00:00 2026-02-25T08:07:00.658 INFO:teuthology.orchestra.run.trial158.stdout:(30/37): librgw2-20.3.0-5338.g6aeb6975.el9.x86_ 2.4 MB/s | 5.6 MB 00:02 2026-02-25T08:07:00.674 INFO:teuthology.orchestra.run.trial006.stdout:(7/37): ceph-common-20.3.0-5338.g6aeb6975.el9.x 3.5 MB/s | 21 MB 00:05 2026-02-25T08:07:00.688 INFO:teuthology.orchestra.run.trial158.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 1.5 MB/s | 67 kB 00:00 2026-02-25T08:07:00.733 INFO:teuthology.orchestra.run.trial158.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 4.1 MB/s | 191 kB 00:00 2026-02-25T08:07:00.735 INFO:teuthology.orchestra.run.trial006.stdout:(8/37): python3-ceph-argparse-20.3.0-5338.g6aeb 747 kB/s | 45 kB 00:00 2026-02-25T08:07:00.829 INFO:teuthology.orchestra.run.trial158.stdout:(33/37): thrift-0.15.0-4.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2026-02-25T08:07:00.864 INFO:teuthology.orchestra.run.trial006.stdout:(9/37): python3-ceph-common-20.3.0-5338.g6aeb69 1.4 MB/s | 185 kB 00:00 2026-02-25T08:07:01.004 INFO:teuthology.orchestra.run.trial006.stdout:(10/37): python3-cephfs-20.3.0-5338.g6aeb6975.e 1.4 MB/s | 204 kB 00:00 2026-02-25T08:07:01.018 INFO:teuthology.orchestra.run.trial158.stdout:(34/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2026-02-25T08:07:01.211 INFO:teuthology.orchestra.run.trial006.stdout:(11/37): python3-rados-20.3.0-5338.g6aeb6975.el 1.5 MB/s | 324 kB 00:00 2026-02-25T08:07:01.404 INFO:teuthology.orchestra.run.trial006.stdout:(12/37): python3-rbd-20.3.0-5338.g6aeb6975.el9. 1.5 MB/s | 306 kB 00:00 2026-02-25T08:07:01.474 INFO:teuthology.orchestra.run.trial006.stdout:(13/37): python3-rgw-20.3.0-5338.g6aeb6975.el9. 1.4 MB/s | 100 kB 00:00 2026-02-25T08:07:01.738 INFO:teuthology.orchestra.run.trial006.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 126 kB/s | 33 kB 00:00 2026-02-25T08:07:01.808 INFO:teuthology.orchestra.run.trial006.stdout:(15/37): ceph-radosgw-20.3.0-5338.g6aeb6975.el9 1.6 MB/s | 11 MB 00:07 2026-02-25T08:07:02.080 INFO:teuthology.orchestra.run.trial158.stdout:(35/37): librbd1-20.3.0-5338.g6aeb6975.el9.x86_ 2.5 MB/s | 2.7 MB 00:01 2026-02-25T08:07:02.098 INFO:teuthology.orchestra.run.trial158.stdout:(36/37): librados2-20.3.0-5338.g6aeb6975.el9.x8 2.6 MB/s | 3.3 MB 00:01 2026-02-25T08:07:02.197 INFO:teuthology.orchestra.run.trial006.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 240 kB/s | 93 kB 00:00 2026-02-25T08:07:02.227 INFO:teuthology.orchestra.run.trial006.stdout:(17/37): nvme-cli-2.16-1.el9.x86_64.rpm 2.4 MB/s | 1.2 MB 00:00 2026-02-25T08:07:02.699 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] boost-program-options-1.75.0-13.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-13.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:02.730 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libnbd-1.20.3-4.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libnbd-1.20.3-4.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:03.366 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] boost-program-options-1.75.0-13.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-13.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:03.396 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libnbd-1.20.3-4.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libnbd-1.20.3-4.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:03.545 INFO:teuthology.orchestra.run.trial158.stdout:(37/37): ceph-common-20.3.0-5338.g6aeb6975.el9. 2.4 MB/s | 21 MB 00:08 2026-02-25T08:07:03.547 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:07:03.547 INFO:teuthology.orchestra.run.trial158.stdout:Total 7.0 MB/s | 61 MB 00:08 2026-02-25T08:07:03.554 INFO:teuthology.orchestra.run.trial006.stdout:(18/37): librgw2-20.3.0-5338.g6aeb6975.el9.x86_ 1.7 MB/s | 5.6 MB 00:03 2026-02-25T08:07:03.643 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:07:03.662 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:07:03.662 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:07:03.704 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] boost-program-options-1.75.0-13.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-13.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:03.731 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libnbd-1.20.3-4.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libnbd-1.20.3-4.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:03.909 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:07:03.909 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:07:03.971 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libpmemobj-1.12.1-1.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libpmemobj-1.12.1-1.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:04.061 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] boost-program-options-1.75.0-13.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-13.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:04.069 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libnbd-1.20.3-4.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libnbd-1.20.3-4.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:04.143 INFO:teuthology.orchestra.run.trial006.stdout:(19/37): boost-program-options-1.75.0-13.el9.x8 53 kB/s | 104 kB 00:01 2026-02-25T08:07:04.200 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:07:04.254 INFO:teuthology.orchestra.run.trial006.stdout:(20/37): libnbd-1.20.3-4.el9.x86_64.rpm 81 kB/s | 164 kB 00:02 2026-02-25T08:07:04.341 INFO:teuthology.orchestra.run.trial158.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-25T08:07:04.346 INFO:teuthology.orchestra.run.trial158.stdout: Upgrading : librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:07:04.388 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libpmemobj-1.12.1-1.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libpmemobj-1.12.1-1.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:04.394 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:07:04.395 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:07:04.407 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:07:04.409 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_6 4/39 2026-02-25T08:07:04.409 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:07:04.438 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:07:04.440 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 6/39 2026-02-25T08:07:04.448 INFO:teuthology.orchestra.run.trial158.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-25T08:07:04.451 INFO:teuthology.orchestra.run.trial158.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-25T08:07:04.454 INFO:teuthology.orchestra.run.trial158.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-25T08:07:04.459 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-25T08:07:04.474 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9. 11/39 2026-02-25T08:07:04.487 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/39 2026-02-25T08:07:04.487 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 13/39 2026-02-25T08:07:04.510 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 13/39 2026-02-25T08:07:04.513 INFO:teuthology.orchestra.run.trial158.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-25T08:07:04.543 INFO:teuthology.orchestra.run.trial158.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-25T08:07:04.547 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-25T08:07:04.553 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-25T08:07:04.555 INFO:teuthology.orchestra.run.trial158.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-25T08:07:04.557 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] librabbitmq-0.11.0-7.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librabbitmq-0.11.0-7.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:04.559 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-25T08:07:04.671 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] librdkafka-1.6.1-102.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librdkafka-1.6.1-102.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:04.673 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T08:07:04.674 INFO:teuthology.orchestra.run.trial158.stdout: Upgrading : librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 21/39 2026-02-25T08:07:04.702 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 21/39 2026-02-25T08:07:04.705 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 22/39 2026-02-25T08:07:04.713 INFO:teuthology.orchestra.run.trial158.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-25T08:07:04.722 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libpmemobj-1.12.1-1.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libpmemobj-1.12.1-1.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:04.963 INFO:teuthology.orchestra.run.trial158.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-25T08:07:04.964 INFO:teuthology.orchestra.run.trial158.stdout: Installing : librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 25/39 2026-02-25T08:07:04.975 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] librabbitmq-0.11.0-7.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librabbitmq-0.11.0-7.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:04.981 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 25/39 2026-02-25T08:07:04.985 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 26/39 2026-02-25T08:07:04.989 INFO:teuthology.orchestra.run.trial158.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-25T08:07:04.991 INFO:teuthology.orchestra.run.trial158.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-25T08:07:04.992 INFO:teuthology.orchestra.run.trial158.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T08:07:05.010 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T08:07:05.063 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libpmemobj-1.12.1-1.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libpmemobj-1.12.1-1.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:05.087 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] librdkafka-1.6.1-102.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librdkafka-1.6.1-102.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:05.125 INFO:teuthology.orchestra.run.trial158.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-25T08:07:05.127 INFO:teuthology.orchestra.run.trial158.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T08:07:05.148 INFO:teuthology.orchestra.run.trial006.stdout:(21/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 101 kB/s | 160 kB 00:01 2026-02-25T08:07:05.311 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] librabbitmq-0.11.0-7.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librabbitmq-0.11.0-7.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:05.391 INFO:teuthology.orchestra.run.trial006.stdout:(22/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 36 kB/s | 45 kB 00:01 2026-02-25T08:07:05.404 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T08:07:05.404 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-25T08:07:05.404 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:07:05.423 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] librdkafka-1.6.1-102.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librdkafka-1.6.1-102.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:05.446 INFO:teuthology.orchestra.run.trial158.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-25T08:07:05.449 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x8 33/39 2026-02-25T08:07:05.479 INFO:teuthology.orchestra.run.trial006.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 2.6 MB/s | 240 kB 00:00 2026-02-25T08:07:05.486 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] lttng-ust-2.12.0-6.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/lttng-ust-2.12.0-6.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:05.558 INFO:teuthology.orchestra.run.trial006.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 470 kB/s | 37 kB 00:00 2026-02-25T08:07:05.639 INFO:teuthology.orchestra.run.trial006.stdout:(25/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 822 kB/s | 66 kB 00:00 2026-02-25T08:07:05.662 INFO:teuthology.orchestra.run.trial006.stdout:(26/37): librdkafka-1.6.1-102.el9.x86_64.rpm 471 kB/s | 662 kB 00:01 2026-02-25T08:07:05.741 INFO:teuthology.orchestra.run.trial006.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.9 MB/s | 308 kB 00:00 2026-02-25T08:07:05.756 INFO:teuthology.orchestra.run.trial006.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.7 MB/s | 25 kB 00:00 2026-02-25T08:07:05.771 INFO:teuthology.orchestra.run.trial006.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 3.2 MB/s | 49 kB 00:00 2026-02-25T08:07:05.786 INFO:teuthology.orchestra.run.trial006.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 4.3 MB/s | 67 kB 00:00 2026-02-25T08:07:05.812 INFO:teuthology.orchestra.run.trial006.stdout:(31/37): libarrow-9.0.0-15.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2026-02-25T08:07:05.827 INFO:teuthology.orchestra.run.trial006.stdout:(32/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 20 MB/s | 838 kB 00:00 2026-02-25T08:07:05.828 INFO:teuthology.orchestra.run.trial006.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 11 MB/s | 191 kB 00:00 2026-02-25T08:07:05.873 INFO:teuthology.orchestra.run.trial006.stdout:(34/37): thrift-0.15.0-4.el9.x86_64.rpm 34 MB/s | 1.6 MB 00:00 2026-02-25T08:07:05.999 INFO:teuthology.orchestra.run.trial006.stdout:(35/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 343 kB/s | 292 kB 00:00 2026-02-25T08:07:06.390 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:07:06.393 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:07:06.652 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:07:06.656 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 35/39 2026-02-25T08:07:06.696 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 35/39 2026-02-25T08:07:06.696 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T08:07:06.696 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T08:07:06.696 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:07:06.699 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:07:07.288 INFO:teuthology.orchestra.run.trial006.stdout:(36/37): librbd1-20.3.0-5338.g6aeb6975.el9.x86_ 1.9 MB/s | 2.7 MB 00:01 2026-02-25T08:07:11.255 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:07:11.255 INFO:teuthology.orchestra.run.trial158.stdout:skipping the directory /sys 2026-02-25T08:07:11.255 INFO:teuthology.orchestra.run.trial158.stdout:skipping the directory /proc 2026-02-25T08:07:11.255 INFO:teuthology.orchestra.run.trial158.stdout:skipping the directory /mnt 2026-02-25T08:07:11.255 INFO:teuthology.orchestra.run.trial158.stdout:skipping the directory /var/tmp 2026-02-25T08:07:11.255 INFO:teuthology.orchestra.run.trial158.stdout:skipping the directory /home 2026-02-25T08:07:11.255 INFO:teuthology.orchestra.run.trial158.stdout:skipping the directory /root 2026-02-25T08:07:11.255 INFO:teuthology.orchestra.run.trial158.stdout:skipping the directory /tmp 2026-02-25T08:07:11.255 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:07:11.514 INFO:teuthology.orchestra.run.trial006.stdout:(37/37): librados2-20.3.0-5338.g6aeb6975.el9.x8 591 kB/s | 3.3 MB 00:05 2026-02-25T08:07:11.516 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:07:11.516 INFO:teuthology.orchestra.run.trial006.stdout:Total 3.6 MB/s | 61 MB 00:17 2026-02-25T08:07:11.615 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:07:11.634 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:07:11.634 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:07:11.746 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 37/39 2026-02-25T08:07:11.769 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 37/39 2026-02-25T08:07:11.769 INFO:teuthology.orchestra.run.trial158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:07:11.769 INFO:teuthology.orchestra.run.trial158.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T08:07:11.769 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T08:07:11.769 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T08:07:11.769 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:07:11.771 INFO:teuthology.orchestra.run.trial158.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T08:07:11.784 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T08:07:11.784 INFO:teuthology.orchestra.run.trial158.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:07:11.837 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:07:11.837 INFO:teuthology.orchestra.run.trial158.stdout:Creating group 'qat' with GID 992. 2026-02-25T08:07:11.838 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:07:11.879 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:07:11.879 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 4/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 6/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 7/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 8/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 9/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9. 10/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x8 11/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 13/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 14/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 15/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-25T08:07:12.127 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-25T08:07:12.128 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-25T08:07:12.128 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-25T08:07:12.128 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:07:12.128 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-25T08:07:12.128 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 38/39 2026-02-25T08:07:12.171 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout:Upgraded: 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T08:07:12.197 INFO:teuthology.orchestra.run.trial158.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:07:12.198 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:07:12.237 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-test 2026-02-25T08:07:12.311 INFO:teuthology.orchestra.run.trial006.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-25T08:07:12.315 INFO:teuthology.orchestra.run.trial006.stdout: Upgrading : librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:07:12.363 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:07:12.364 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:07:12.377 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:07:12.379 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_6 4/39 2026-02-25T08:07:12.380 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:07:12.408 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:07:12.411 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 6/39 2026-02-25T08:07:12.418 INFO:teuthology.orchestra.run.trial006.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-25T08:07:12.421 INFO:teuthology.orchestra.run.trial006.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-25T08:07:12.424 INFO:teuthology.orchestra.run.trial006.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-25T08:07:12.428 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-25T08:07:12.439 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9. 11/39 2026-02-25T08:07:12.452 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/39 2026-02-25T08:07:12.452 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 13/39 2026-02-25T08:07:12.479 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 13/39 2026-02-25T08:07:12.483 INFO:teuthology.orchestra.run.trial006.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-25T08:07:12.518 INFO:teuthology.orchestra.run.trial006.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-25T08:07:12.521 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-25T08:07:12.527 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-25T08:07:12.530 INFO:teuthology.orchestra.run.trial006.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-25T08:07:12.534 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-25T08:07:12.556 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:19 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: Package Arch Version Repository Size 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: ceph-test x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 127 M 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout:Installing dependencies: 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: libcephsqlite x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 153 k 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T08:07:12.642 INFO:teuthology.orchestra.run.trial158.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-25T08:07:12.643 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:07:12.643 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:07:12.643 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:07:12.643 INFO:teuthology.orchestra.run.trial158.stdout:Install 12 Packages 2026-02-25T08:07:12.643 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:07:12.643 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 132 M 2026-02-25T08:07:12.643 INFO:teuthology.orchestra.run.trial158.stdout:Installed size: 988 M 2026-02-25T08:07:12.643 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:07:12.647 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T08:07:12.648 INFO:teuthology.orchestra.run.trial006.stdout: Upgrading : librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 21/39 2026-02-25T08:07:12.674 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 21/39 2026-02-25T08:07:12.676 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 22/39 2026-02-25T08:07:12.683 INFO:teuthology.orchestra.run.trial006.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-25T08:07:12.930 INFO:teuthology.orchestra.run.trial006.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-25T08:07:12.931 INFO:teuthology.orchestra.run.trial006.stdout: Installing : librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 25/39 2026-02-25T08:07:12.947 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 25/39 2026-02-25T08:07:12.951 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 26/39 2026-02-25T08:07:12.955 INFO:teuthology.orchestra.run.trial006.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-25T08:07:12.957 INFO:teuthology.orchestra.run.trial006.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-25T08:07:12.958 INFO:teuthology.orchestra.run.trial006.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T08:07:12.977 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T08:07:13.090 INFO:teuthology.orchestra.run.trial006.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-25T08:07:13.092 INFO:teuthology.orchestra.run.trial006.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T08:07:13.333 INFO:teuthology.orchestra.run.trial158.stdout:(1/12): libcephsqlite-20.3.0-5338.g6aeb6975.el9 328 kB/s | 153 kB 00:00 2026-02-25T08:07:13.380 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T08:07:13.380 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-25T08:07:13.380 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:07:13.424 INFO:teuthology.orchestra.run.trial006.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-25T08:07:13.426 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x8 33/39 2026-02-25T08:07:13.578 INFO:teuthology.orchestra.run.trial158.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 93 kB/s | 23 kB 00:00 2026-02-25T08:07:13.606 INFO:teuthology.orchestra.run.trial158.stdout:(3/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.8 MB/s | 2.1 MB 00:00 2026-02-25T08:07:13.812 INFO:teuthology.orchestra.run.trial158.stdout:(4/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 400 kB/s | 94 kB 00:00 2026-02-25T08:07:13.815 INFO:teuthology.orchestra.run.trial158.stdout:(5/12): libasan8-15.2.1-7.el9.x86_64.rpm 2.4 MB/s | 525 kB 00:00 2026-02-25T08:07:13.846 INFO:teuthology.orchestra.run.trial158.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 7.4 MB/s | 233 kB 00:00 2026-02-25T08:07:13.904 INFO:teuthology.orchestra.run.trial158.stdout:(7/12): protobuf-3.14.0-17.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2026-02-25T08:07:13.934 INFO:teuthology.orchestra.run.trial158.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 9.8 MB/s | 303 kB 00:00 2026-02-25T08:07:13.960 INFO:teuthology.orchestra.run.trial158.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.4 MB/s | 64 kB 00:00 2026-02-25T08:07:13.965 INFO:teuthology.orchestra.run.trial158.stdout:(10/12): libubsan-11.5.0-14.el9.x86_64.rpm 1.1 MB/s | 176 kB 00:00 2026-02-25T08:07:14.102 INFO:teuthology.orchestra.run.trial158.stdout:(11/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 859 kB/s | 122 kB 00:00 2026-02-25T08:07:14.366 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:07:14.368 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:07:14.630 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:07:14.633 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 35/39 2026-02-25T08:07:14.674 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 35/39 2026-02-25T08:07:14.674 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T08:07:14.674 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T08:07:14.674 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:07:14.677 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:07:19.243 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:07:19.243 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /sys 2026-02-25T08:07:19.243 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /proc 2026-02-25T08:07:19.243 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /mnt 2026-02-25T08:07:19.243 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /var/tmp 2026-02-25T08:07:19.243 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /home 2026-02-25T08:07:19.243 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /root 2026-02-25T08:07:19.244 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /tmp 2026-02-25T08:07:19.244 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:07:19.738 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 37/39 2026-02-25T08:07:19.762 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 37/39 2026-02-25T08:07:19.762 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:07:19.762 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T08:07:19.762 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T08:07:19.762 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T08:07:19.762 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:07:19.764 INFO:teuthology.orchestra.run.trial006.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T08:07:19.776 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T08:07:19.776 INFO:teuthology.orchestra.run.trial006.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:07:19.831 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:07:19.832 INFO:teuthology.orchestra.run.trial006.stdout:Creating group 'qat' with GID 992. 2026-02-25T08:07:19.832 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 4/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 6/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 7/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 8/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 9/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9. 10/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x8 11/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 13/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 14/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 15/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T08:07:20.120 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-25T08:07:20.121 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 38/39 2026-02-25T08:07:20.188 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout:Upgraded: 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T08:07:20.189 INFO:teuthology.orchestra.run.trial006.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T08:07:20.190 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:07:20.190 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:07:20.229 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-test 2026-02-25T08:07:20.550 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:27 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:07:20.636 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repository Size 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: ceph-test x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 127 M 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout:Installing dependencies: 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: libcephsqlite x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 153 k 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout:Install 12 Packages 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:07:20.637 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 132 M 2026-02-25T08:07:20.638 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 988 M 2026-02-25T08:07:20.638 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:07:21.886 INFO:teuthology.orchestra.run.trial006.stdout:(1/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.9 MB/s | 2.1 MB 00:00 2026-02-25T08:07:21.913 INFO:teuthology.orchestra.run.trial006.stdout:(2/12): libcephsqlite-20.3.0-5338.g6aeb6975.el9 201 kB/s | 153 kB 00:00 2026-02-25T08:07:21.965 INFO:teuthology.orchestra.run.trial006.stdout:(3/12): libatomic-11.5.0-14.el9.x86_64.rpm 293 kB/s | 23 kB 00:00 2026-02-25T08:07:22.308 INFO:teuthology.orchestra.run.trial006.stdout:(4/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 238 kB/s | 94 kB 00:00 2026-02-25T08:07:22.337 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libasan8-15.2.1-7.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libasan8-15.2.1-7.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:22.666 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libubsan-11.5.0-14.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libubsan-11.5.0-14.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:22.758 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libasan8-15.2.1-7.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libasan8-15.2.1-7.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:22.813 INFO:teuthology.orchestra.run.trial006.stdout:(5/12): libasan8-15.2.1-7.el9.x86_64.rpm 618 kB/s | 525 kB 00:00 2026-02-25T08:07:23.019 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libubsan-11.5.0-14.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libubsan-11.5.0-14.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:23.044 INFO:teuthology.orchestra.run.trial006.stdout:(6/12): libubsan-11.5.0-14.el9.x86_64.rpm 239 kB/s | 176 kB 00:00 2026-02-25T08:07:23.170 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libxslt-1.1.34-12.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libxslt-1.1.34-12.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:23.398 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] protobuf-3.14.0-17.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/protobuf-3.14.0-17.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:23.522 INFO:teuthology.orchestra.run.trial006.stdout:[MIRROR] libxslt-1.1.34-12.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libxslt-1.1.34-12.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:07:23.547 INFO:teuthology.orchestra.run.trial006.stdout:(7/12): libxslt-1.1.34-12.el9.x86_64.rpm 318 kB/s | 233 kB 00:00 2026-02-25T08:07:23.567 INFO:teuthology.orchestra.run.trial006.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 15 MB/s | 303 kB 00:00 2026-02-25T08:07:23.587 INFO:teuthology.orchestra.run.trial006.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 3.1 MB/s | 64 kB 00:00 2026-02-25T08:07:23.664 INFO:teuthology.orchestra.run.trial006.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 1.6 MB/s | 122 kB 00:00 2026-02-25T08:07:25.061 INFO:teuthology.orchestra.run.trial006.stdout:(11/12): protobuf-3.14.0-17.el9.x86_64.rpm 510 kB/s | 1.0 MB 00:02 2026-02-25T08:10:32.418 INFO:teuthology.orchestra.run.trial006.stdout:(12/12): ceph-test-20.3.0-5338.g6aeb6975.el9.x8 679 kB/s | 127 MB 03:11 2026-02-25T08:10:32.421 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:10:32.421 INFO:teuthology.orchestra.run.trial006.stdout:Total 703 kB/s | 132 MB 03:11 2026-02-25T08:10:32.494 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:10:32.507 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:10:32.507 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:10:32.939 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:10:32.940 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:10:33.398 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:10:33.407 INFO:teuthology.orchestra.run.trial006.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-25T08:10:33.416 INFO:teuthology.orchestra.run.trial006.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-25T08:10:33.421 INFO:teuthology.orchestra.run.trial006.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-25T08:10:33.425 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-25T08:10:33.427 INFO:teuthology.orchestra.run.trial006.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-25T08:10:33.432 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-25T08:10:33.433 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T08:10:33.449 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T08:10:33.451 INFO:teuthology.orchestra.run.trial006.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-25T08:10:33.461 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-25T08:10:33.469 INFO:teuthology.orchestra.run.trial006.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-25T08:10:33.470 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 11/12 2026-02-25T08:10:40.421 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 11/12 2026-02-25T08:10:40.441 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/12 2026-02-25T08:10:41.232 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/12 2026-02-25T08:10:41.232 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/12 2026-02-25T08:10:41.232 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/12 2026-02-25T08:10:41.232 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-25T08:10:41.232 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-25T08:10:41.232 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-25T08:10:41.232 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-25T08:10:41.232 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-25T08:10:41.233 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-25T08:10:41.233 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-25T08:10:41.233 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-25T08:10:41.233 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-25T08:10:41.290 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-25T08:10:41.290 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:10:41.290 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:10:41.290 INFO:teuthology.orchestra.run.trial006.stdout: ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:10:41.290 INFO:teuthology.orchestra.run.trial006.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-25T08:10:41.290 INFO:teuthology.orchestra.run.trial006.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-25T08:10:41.291 INFO:teuthology.orchestra.run.trial006.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-25T08:10:41.291 INFO:teuthology.orchestra.run.trial006.stdout: libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:10:41.291 INFO:teuthology.orchestra.run.trial006.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-25T08:10:41.291 INFO:teuthology.orchestra.run.trial006.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T08:10:41.291 INFO:teuthology.orchestra.run.trial006.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-25T08:10:41.291 INFO:teuthology.orchestra.run.trial006.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T08:10:41.291 INFO:teuthology.orchestra.run.trial006.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T08:10:41.291 INFO:teuthology.orchestra.run.trial006.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T08:10:41.291 INFO:teuthology.orchestra.run.trial006.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-25T08:10:41.291 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:10:41.291 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:10:41.332 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph 2026-02-25T08:10:41.643 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:03:48 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:10:41.730 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:10:41.733 INFO:teuthology.orchestra.run.trial006.stdout:======================================================================================= 2026-02-25T08:10:41.733 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repository Size 2026-02-25T08:10:41.733 INFO:teuthology.orchestra.run.trial006.stdout:======================================================================================= 2026-02-25T08:10:41.733 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:10:41.733 INFO:teuthology.orchestra.run.trial006.stdout: ceph x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 6.5 k 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout:Installing dependencies: 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 57 k 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mds x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.0 M 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 937 k 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 297 k 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mon x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 4.9 M 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 163 k 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd-classic x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 15 M 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd-crimson x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 88 M 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 17 k 2026-02-25T08:10:41.734 INFO:teuthology.orchestra.run.trial006.stdout: cephadm noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 1.0 M 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T08:10:41.735 INFO:teuthology.orchestra.run.trial006.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T08:10:41.736 INFO:teuthology.orchestra.run.trial006.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T08:10:41.737 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T08:10:41.738 INFO:teuthology.orchestra.run.trial006.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T08:10:41.739 INFO:teuthology.orchestra.run.trial006.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T08:10:41.740 INFO:teuthology.orchestra.run.trial006.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout:Installing weak dependencies: 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 184 k 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 51 M 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 7.4 M 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 22 k 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-rook noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 50 k 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout: ceph-volume noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 301 k 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-25T08:10:41.741 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:10:41.742 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:10:41.742 INFO:teuthology.orchestra.run.trial006.stdout:======================================================================================= 2026-02-25T08:10:41.742 INFO:teuthology.orchestra.run.trial006.stdout:Install 96 Packages 2026-02-25T08:10:41.742 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:10:41.742 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 226 M 2026-02-25T08:10:41.743 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 1.5 G 2026-02-25T08:10:41.743 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:10:42.587 INFO:teuthology.orchestra.run.trial006.stdout:(1/96): ceph-20.3.0-5338.g6aeb6975.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2026-02-25T08:10:43.290 INFO:teuthology.orchestra.run.trial006.stdout:(2/96): ceph-mgr-20.3.0-5338.g6aeb6975.el9.x86_ 932 kB/s | 937 kB 00:01 2026-02-25T08:10:43.429 INFO:teuthology.orchestra.run.trial006.stdout:(3/96): ceph-osd-20.3.0-5338.g6aeb6975.el9.x86_ 1.2 MB/s | 163 kB 00:00 2026-02-25T08:10:44.401 INFO:teuthology.orchestra.run.trial006.stdout:(4/96): ceph-mds-20.3.0-5338.g6aeb6975.el9.x86_ 978 kB/s | 2.0 MB 00:02 2026-02-25T08:10:49.833 INFO:teuthology.orchestra.run.trial006.stdout:(5/96): ceph-mon-20.3.0-5338.g6aeb6975.el9.x86_ 688 kB/s | 4.9 MB 00:07 2026-02-25T08:10:49.903 INFO:teuthology.orchestra.run.trial006.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5338.g6a 820 kB/s | 57 kB 00:00 2026-02-25T08:10:50.184 INFO:teuthology.orchestra.run.trial006.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5338.g6aeb6975. 655 kB/s | 184 kB 00:00 2026-02-25T08:11:12.164 INFO:teuthology.orchestra.run.trial006.stdout:(8/96): ceph-osd-classic-20.3.0-5338.g6aeb6975. 526 kB/s | 15 MB 00:28 2026-02-25T08:11:30.322 INFO:teuthology.orchestra.run.trial006.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-53 418 kB/s | 7.4 MB 00:18 2026-02-25T08:11:30.391 INFO:teuthology.orchestra.run.trial006.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5338.g6aeb69 323 kB/s | 22 kB 00:00 2026-02-25T08:11:30.700 INFO:teuthology.orchestra.run.trial158.stdout:(12/12): ceph-test-20.3.0-5338.g6aeb6975.el9.x8 504 kB/s | 127 MB 04:17 2026-02-25T08:11:30.702 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:11:30.702 INFO:teuthology.orchestra.run.trial158.stdout:Total 522 kB/s | 132 MB 04:18 2026-02-25T08:11:30.771 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:11:30.782 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:11:30.782 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:11:31.088 INFO:teuthology.orchestra.run.trial006.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5338.g6ae 426 kB/s | 297 kB 00:00 2026-02-25T08:11:31.215 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:11:31.215 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:11:31.225 INFO:teuthology.orchestra.run.trial006.stdout:(12/96): ceph-mgr-rook-20.3.0-5338.g6aeb6975.el 366 kB/s | 50 kB 00:00 2026-02-25T08:11:31.296 INFO:teuthology.orchestra.run.trial006.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5338.g6a 243 kB/s | 17 kB 00:00 2026-02-25T08:11:31.673 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:11:31.681 INFO:teuthology.orchestra.run.trial158.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-25T08:11:31.690 INFO:teuthology.orchestra.run.trial158.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-25T08:11:31.695 INFO:teuthology.orchestra.run.trial158.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-25T08:11:31.699 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-25T08:11:31.701 INFO:teuthology.orchestra.run.trial158.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-25T08:11:31.706 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-25T08:11:31.706 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T08:11:31.721 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T08:11:31.723 INFO:teuthology.orchestra.run.trial158.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-25T08:11:31.733 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-25T08:11:31.742 INFO:teuthology.orchestra.run.trial158.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-25T08:11:31.743 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 11/12 2026-02-25T08:11:31.926 INFO:teuthology.orchestra.run.trial006.stdout:(14/96): ceph-volume-20.3.0-5338.g6aeb6975.el9. 479 kB/s | 301 kB 00:00 2026-02-25T08:11:35.270 INFO:teuthology.orchestra.run.trial006.stdout:(15/96): cephadm-20.3.0-5338.g6aeb6975.el9.noar 311 kB/s | 1.0 MB 00:03 2026-02-25T08:11:36.278 INFO:teuthology.orchestra.run.trial006.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 348 kB/s | 351 kB 00:01 2026-02-25T08:11:36.456 INFO:teuthology.orchestra.run.trial006.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 227 kB/s | 40 kB 00:00 2026-02-25T08:11:36.702 INFO:teuthology.orchestra.run.trial006.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 293 kB/s | 72 kB 00:00 2026-02-25T08:11:37.764 INFO:teuthology.orchestra.run.trial006.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 748 kB/s | 794 kB 00:01 2026-02-25T08:11:38.092 INFO:teuthology.orchestra.run.trial006.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 563 kB/s | 184 kB 00:00 2026-02-25T08:11:38.501 INFO:teuthology.orchestra.run.trial006.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 618 kB/s | 253 kB 00:00 2026-02-25T08:11:38.786 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 11/12 2026-02-25T08:11:38.804 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/12 2026-02-25T08:11:39.567 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/12 2026-02-25T08:11:39.567 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/12 2026-02-25T08:11:39.567 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/12 2026-02-25T08:11:39.567 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-25T08:11:39.567 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-25T08:11:39.567 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-25T08:11:39.567 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-25T08:11:39.567 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-25T08:11:39.568 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-25T08:11:39.568 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-25T08:11:39.568 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-25T08:11:39.568 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-25T08:11:39.626 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-25T08:11:39.626 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:11:39.626 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:11:39.626 INFO:teuthology.orchestra.run.trial158.stdout: ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:11:39.626 INFO:teuthology.orchestra.run.trial158.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout: libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:11:39.627 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:11:39.667 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph 2026-02-25T08:11:39.903 INFO:teuthology.orchestra.run.trial006.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 910 kB/s | 1.2 MB 00:01 2026-02-25T08:11:39.976 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:04:46 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:11:40.063 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:11:40.066 INFO:teuthology.orchestra.run.trial158.stdout:======================================================================================= 2026-02-25T08:11:40.066 INFO:teuthology.orchestra.run.trial158.stdout: Package Arch Version Repository Size 2026-02-25T08:11:40.066 INFO:teuthology.orchestra.run.trial158.stdout:======================================================================================= 2026-02-25T08:11:40.067 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:11:40.067 INFO:teuthology.orchestra.run.trial158.stdout: ceph x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 6.5 k 2026-02-25T08:11:40.067 INFO:teuthology.orchestra.run.trial158.stdout:Installing dependencies: 2026-02-25T08:11:40.067 INFO:teuthology.orchestra.run.trial158.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T08:11:40.067 INFO:teuthology.orchestra.run.trial158.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 57 k 2026-02-25T08:11:40.067 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mds x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.0 M 2026-02-25T08:11:40.067 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 937 k 2026-02-25T08:11:40.067 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 297 k 2026-02-25T08:11:40.067 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mon x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 4.9 M 2026-02-25T08:11:40.067 INFO:teuthology.orchestra.run.trial158.stdout: ceph-osd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 163 k 2026-02-25T08:11:40.067 INFO:teuthology.orchestra.run.trial158.stdout: ceph-osd-classic x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 15 M 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: ceph-osd-crimson x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 88 M 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 17 k 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: cephadm noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 1.0 M 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T08:11:40.068 INFO:teuthology.orchestra.run.trial158.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T08:11:40.069 INFO:teuthology.orchestra.run.trial158.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T08:11:40.070 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T08:11:40.071 INFO:teuthology.orchestra.run.trial158.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T08:11:40.072 INFO:teuthology.orchestra.run.trial158.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-25T08:11:40.073 INFO:teuthology.orchestra.run.trial158.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout:Installing weak dependencies: 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 184 k 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 51 M 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 7.4 M 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 22 k 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-rook noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 50 k 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout: ceph-volume noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 301 k 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:11:40.074 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:11:40.075 INFO:teuthology.orchestra.run.trial158.stdout:======================================================================================= 2026-02-25T08:11:40.075 INFO:teuthology.orchestra.run.trial158.stdout:Install 96 Packages 2026-02-25T08:11:40.075 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:11:40.075 INFO:teuthology.orchestra.run.trial006.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 645 kB/s | 106 kB 00:00 2026-02-25T08:11:40.076 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 226 M 2026-02-25T08:11:40.076 INFO:teuthology.orchestra.run.trial158.stdout:Installed size: 1.5 G 2026-02-25T08:11:40.076 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:11:40.234 INFO:teuthology.orchestra.run.trial006.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 816 kB/s | 135 kB 00:00 2026-02-25T08:11:40.400 INFO:teuthology.orchestra.run.trial006.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 909 kB/s | 150 kB 00:00 2026-02-25T08:11:40.566 INFO:teuthology.orchestra.run.trial006.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 761 kB/s | 126 kB 00:00 2026-02-25T08:11:40.649 INFO:teuthology.orchestra.run.trial158.stdout:(1/96): ceph-20.3.0-5338.g6aeb6975.el9.x86_64.r 23 kB/s | 6.5 kB 00:00 2026-02-25T08:11:40.732 INFO:teuthology.orchestra.run.trial006.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 1.3 MB/s | 218 kB 00:00 2026-02-25T08:11:40.897 INFO:teuthology.orchestra.run.trial006.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 1.1 MB/s | 182 kB 00:00 2026-02-25T08:11:41.063 INFO:teuthology.orchestra.run.trial006.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 1.6 MB/s | 266 kB 00:00 2026-02-25T08:11:41.156 INFO:teuthology.orchestra.run.trial006.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 317 kB/s | 30 kB 00:00 2026-02-25T08:11:41.345 INFO:teuthology.orchestra.run.trial006.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 16 MB/s | 3.0 MB 00:00 2026-02-25T08:11:41.369 INFO:teuthology.orchestra.run.trial006.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 588 kB/s | 15 kB 00:00 2026-02-25T08:11:41.396 INFO:teuthology.orchestra.run.trial006.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 8.9 MB/s | 246 kB 00:00 2026-02-25T08:11:41.423 INFO:teuthology.orchestra.run.trial006.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 6.8 MB/s | 188 kB 00:00 2026-02-25T08:11:41.449 INFO:teuthology.orchestra.run.trial006.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 1.6 MB/s | 42 kB 00:00 2026-02-25T08:11:41.621 INFO:teuthology.orchestra.run.trial006.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 31 MB/s | 5.3 MB 00:00 2026-02-25T08:11:41.686 INFO:teuthology.orchestra.run.trial006.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 92 MB/s | 6.0 MB 00:00 2026-02-25T08:11:41.714 INFO:teuthology.orchestra.run.trial006.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 8.5 MB/s | 244 kB 00:00 2026-02-25T08:11:41.742 INFO:teuthology.orchestra.run.trial006.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 8.7 MB/s | 249 kB 00:00 2026-02-25T08:11:41.768 INFO:teuthology.orchestra.run.trial006.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 1.8 MB/s | 48 kB 00:00 2026-02-25T08:11:41.795 INFO:teuthology.orchestra.run.trial006.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 6.4 MB/s | 177 kB 00:00 2026-02-25T08:11:41.826 INFO:teuthology.orchestra.run.trial006.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 39 MB/s | 1.2 MB 00:00 2026-02-25T08:11:41.852 INFO:teuthology.orchestra.run.trial006.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 1.3 MB/s | 35 kB 00:00 2026-02-25T08:11:41.917 INFO:teuthology.orchestra.run.trial006.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 95 MB/s | 6.1 MB 00:00 2026-02-25T08:11:41.946 INFO:teuthology.orchestra.run.trial006.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 15 MB/s | 442 kB 00:00 2026-02-25T08:11:41.973 INFO:teuthology.orchestra.run.trial006.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 2.8 MB/s | 77 kB 00:00 2026-02-25T08:11:42.000 INFO:teuthology.orchestra.run.trial006.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 9.7 MB/s | 267 kB 00:00 2026-02-25T08:11:42.027 INFO:teuthology.orchestra.run.trial006.stdout:(48/96): python3-pyasn1-0.4.8-7.el9.noarch.rpm 5.6 MB/s | 157 kB 00:00 2026-02-25T08:11:42.056 INFO:teuthology.orchestra.run.trial006.stdout:(49/96): python3-pyasn1-modules-0.4.8-7.el9.noa 9.7 MB/s | 277 kB 00:00 2026-02-25T08:11:42.082 INFO:teuthology.orchestra.run.trial006.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 2.0 MB/s | 54 kB 00:00 2026-02-25T08:11:42.262 INFO:teuthology.orchestra.run.trial006.stdout:(51/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 107 MB/s | 19 MB 00:00 2026-02-25T08:11:42.289 INFO:teuthology.orchestra.run.trial006.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 1.6 MB/s | 42 kB 00:00 2026-02-25T08:11:42.315 INFO:teuthology.orchestra.run.trial006.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 6.9 MB/s | 189 kB 00:00 2026-02-25T08:11:42.341 INFO:teuthology.orchestra.run.trial006.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.4 MB/s | 90 kB 00:00 2026-02-25T08:11:42.367 INFO:teuthology.orchestra.run.trial006.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 862 kB/s | 22 kB 00:00 2026-02-25T08:11:42.411 INFO:teuthology.orchestra.run.trial006.stdout:(56/96): protobuf-compiler-3.14.0-17.el9.x86_64 19 MB/s | 862 kB 00:00 2026-02-25T08:11:42.447 INFO:teuthology.orchestra.run.trial006.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 15 MB/s | 551 kB 00:00 2026-02-25T08:11:42.450 INFO:teuthology.orchestra.run.trial006.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 5.7 MB/s | 19 kB 00:00 2026-02-25T08:11:42.454 INFO:teuthology.orchestra.run.trial006.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 38 MB/s | 151 kB 00:00 2026-02-25T08:11:42.459 INFO:teuthology.orchestra.run.trial006.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 113 MB/s | 548 kB 00:00 2026-02-25T08:11:42.463 INFO:teuthology.orchestra.run.trial006.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 8.0 MB/s | 29 kB 00:00 2026-02-25T08:11:42.467 INFO:teuthology.orchestra.run.trial006.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 17 MB/s | 60 kB 00:00 2026-02-25T08:11:42.470 INFO:teuthology.orchestra.run.trial006.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 12 MB/s | 43 kB 00:00 2026-02-25T08:11:42.474 INFO:teuthology.orchestra.run.trial006.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 9.1 MB/s | 32 kB 00:00 2026-02-25T08:11:42.477 INFO:teuthology.orchestra.run.trial006.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 3.9 MB/s | 14 kB 00:00 2026-02-25T08:11:42.481 INFO:teuthology.orchestra.run.trial006.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 44 MB/s | 173 kB 00:00 2026-02-25T08:11:42.486 INFO:teuthology.orchestra.run.trial006.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 82 MB/s | 358 kB 00:00 2026-02-25T08:11:42.490 INFO:teuthology.orchestra.run.trial006.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 63 MB/s | 254 kB 00:00 2026-02-25T08:11:42.500 INFO:teuthology.orchestra.run.trial006.stdout:(69/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 208 MB/s | 2.0 MB 00:00 2026-02-25T08:11:42.504 INFO:teuthology.orchestra.run.trial006.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 37 MB/s | 144 kB 00:00 2026-02-25T08:11:42.508 INFO:teuthology.orchestra.run.trial006.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 36 MB/s | 139 kB 00:00 2026-02-25T08:11:42.511 INFO:teuthology.orchestra.run.trial006.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 16 MB/s | 56 kB 00:00 2026-02-25T08:11:42.515 INFO:teuthology.orchestra.run.trial006.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 3.0 MB/s | 11 kB 00:00 2026-02-25T08:11:42.518 INFO:teuthology.orchestra.run.trial006.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 5.1 MB/s | 18 kB 00:00 2026-02-25T08:11:42.522 INFO:teuthology.orchestra.run.trial006.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 6.6 MB/s | 23 kB 00:00 2026-02-25T08:11:42.526 INFO:teuthology.orchestra.run.trial006.stdout:(76/96): python3-jaraco-context-6.0.1-3.el9.noa 5.1 MB/s | 20 kB 00:00 2026-02-25T08:11:42.529 INFO:teuthology.orchestra.run.trial006.stdout:(77/96): python3-jaraco-functools-3.5.0-2.el9.n 5.3 MB/s | 19 kB 00:00 2026-02-25T08:11:42.533 INFO:teuthology.orchestra.run.trial006.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 7.4 MB/s | 26 kB 00:00 2026-02-25T08:11:42.539 INFO:teuthology.orchestra.run.trial006.stdout:(79/96): python3-kubernetes-26.1.0-3.el9.noarch 179 MB/s | 1.0 MB 00:00 2026-02-25T08:11:42.542 INFO:teuthology.orchestra.run.trial006.stdout:(80/96): python3-more-itertools-8.12.0-2.el9.no 21 MB/s | 79 kB 00:00 2026-02-25T08:11:42.546 INFO:teuthology.orchestra.run.trial006.stdout:(81/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 23 MB/s | 86 kB 00:00 2026-02-25T08:11:42.550 INFO:teuthology.orchestra.run.trial006.stdout:(82/96): python3-natsort-7.1.1-5.el9.noarch.rpm 16 MB/s | 58 kB 00:00 2026-02-25T08:11:42.553 INFO:teuthology.orchestra.run.trial006.stdout:(83/96): python3-portend-3.1.0-2.el9.noarch.rpm 4.7 MB/s | 16 kB 00:00 2026-02-25T08:11:42.557 INFO:teuthology.orchestra.run.trial006.stdout:(84/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 23 MB/s | 90 kB 00:00 2026-02-25T08:11:42.561 INFO:teuthology.orchestra.run.trial006.stdout:(85/96): python3-repoze-lru-0.7-16.el9.noarch.r 8.4 MB/s | 31 kB 00:00 2026-02-25T08:11:42.565 INFO:teuthology.orchestra.run.trial006.stdout:(86/96): python3-routes-2.5.1-5.el9.noarch.rpm 45 MB/s | 188 kB 00:00 2026-02-25T08:11:42.569 INFO:teuthology.orchestra.run.trial006.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 16 MB/s | 59 kB 00:00 2026-02-25T08:11:42.573 INFO:teuthology.orchestra.run.trial006.stdout:(88/96): python3-saml-1.16.0-1.el9.noarch.rpm 33 MB/s | 125 kB 00:00 2026-02-25T08:11:42.577 INFO:teuthology.orchestra.run.trial006.stdout:(89/96): python3-tempora-5.0.0-2.el9.noarch.rpm 9.9 MB/s | 36 kB 00:00 2026-02-25T08:11:42.580 INFO:teuthology.orchestra.run.trial006.stdout:(90/96): python3-typing-extensions-4.15.0-1.el9 24 MB/s | 86 kB 00:00 2026-02-25T08:11:42.584 INFO:teuthology.orchestra.run.trial006.stdout:(91/96): python3-websocket-client-1.2.3-2.el9.n 24 MB/s | 90 kB 00:00 2026-02-25T08:11:42.588 INFO:teuthology.orchestra.run.trial006.stdout:(92/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 13 MB/s | 48 kB 00:00 2026-02-25T08:11:42.591 INFO:teuthology.orchestra.run.trial006.stdout:(93/96): python3-xmltodict-0.12.0-15.el9.noarch 6.3 MB/s | 22 kB 00:00 2026-02-25T08:11:42.595 INFO:teuthology.orchestra.run.trial006.stdout:(94/96): python3-zc-lockfile-2.0-10.el9.noarch. 5.7 MB/s | 20 kB 00:00 2026-02-25T08:11:43.058 INFO:teuthology.orchestra.run.trial158.stdout:(2/96): ceph-mgr-20.3.0-5338.g6aeb6975.el9.x86_ 347 kB/s | 937 kB 00:02 2026-02-25T08:11:43.479 INFO:teuthology.orchestra.run.trial158.stdout:(3/96): ceph-osd-20.3.0-5338.g6aeb6975.el9.x86_ 388 kB/s | 163 kB 00:00 2026-02-25T08:11:48.717 INFO:teuthology.orchestra.run.trial158.stdout:(4/96): ceph-mds-20.3.0-5338.g6aeb6975.el9.x86_ 248 kB/s | 2.0 MB 00:08 2026-02-25T08:11:49.220 INFO:teuthology.orchestra.run.trial158.stdout:(5/96): ceph-mon-20.3.0-5338.g6aeb6975.el9.x86_ 581 kB/s | 4.9 MB 00:08 2026-02-25T08:11:49.501 INFO:teuthology.orchestra.run.trial158.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5338.g6a 203 kB/s | 57 kB 00:00 2026-02-25T08:11:50.273 INFO:teuthology.orchestra.run.trial158.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5338.g6aeb6975. 238 kB/s | 184 kB 00:00 2026-02-25T08:12:14.396 INFO:teuthology.orchestra.run.trial158.stdout:(8/96): ceph-osd-classic-20.3.0-5338.g6aeb6975. 489 kB/s | 15 MB 00:30 2026-02-25T08:12:31.067 INFO:teuthology.orchestra.run.trial158.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-53 455 kB/s | 7.4 MB 00:16 2026-02-25T08:12:31.137 INFO:teuthology.orchestra.run.trial158.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5338.g6aeb69 316 kB/s | 22 kB 00:00 2026-02-25T08:12:31.834 INFO:teuthology.orchestra.run.trial158.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5338.g6ae 426 kB/s | 297 kB 00:00 2026-02-25T08:12:31.975 INFO:teuthology.orchestra.run.trial158.stdout:(12/96): ceph-mgr-rook-20.3.0-5338.g6aeb6975.el 358 kB/s | 50 kB 00:00 2026-02-25T08:12:32.046 INFO:teuthology.orchestra.run.trial158.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5338.g6a 243 kB/s | 17 kB 00:00 2026-02-25T08:12:32.647 INFO:teuthology.orchestra.run.trial158.stdout:(14/96): ceph-volume-20.3.0-5338.g6aeb6975.el9. 501 kB/s | 301 kB 00:00 2026-02-25T08:12:34.802 INFO:teuthology.orchestra.run.trial158.stdout:(15/96): cephadm-20.3.0-5338.g6aeb6975.el9.noar 483 kB/s | 1.0 MB 00:02 2026-02-25T08:12:35.639 INFO:teuthology.orchestra.run.trial158.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 419 kB/s | 351 kB 00:00 2026-02-25T08:12:35.726 INFO:teuthology.orchestra.run.trial158.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 468 kB/s | 40 kB 00:00 2026-02-25T08:12:35.894 INFO:teuthology.orchestra.run.trial158.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 429 kB/s | 72 kB 00:00 2026-02-25T08:12:36.879 INFO:teuthology.orchestra.run.trial158.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 806 kB/s | 794 kB 00:00 2026-02-25T08:12:37.130 INFO:teuthology.orchestra.run.trial158.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 736 kB/s | 184 kB 00:00 2026-02-25T08:12:37.463 INFO:teuthology.orchestra.run.trial158.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 759 kB/s | 253 kB 00:00 2026-02-25T08:12:39.023 INFO:teuthology.orchestra.run.trial158.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 818 kB/s | 1.2 MB 00:01 2026-02-25T08:12:39.192 INFO:teuthology.orchestra.run.trial158.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 629 kB/s | 106 kB 00:00 2026-02-25T08:12:39.336 INFO:teuthology.orchestra.run.trial006.stdout:(95/96): ceph-mgr-dashboard-20.3.0-5338.g6aeb69 475 kB/s | 51 MB 01:49 2026-02-25T08:12:39.361 INFO:teuthology.orchestra.run.trial158.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 800 kB/s | 135 kB 00:00 2026-02-25T08:12:39.611 INFO:teuthology.orchestra.run.trial158.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 601 kB/s | 150 kB 00:00 2026-02-25T08:12:39.780 INFO:teuthology.orchestra.run.trial158.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 749 kB/s | 126 kB 00:00 2026-02-25T08:12:40.113 INFO:teuthology.orchestra.run.trial158.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 654 kB/s | 218 kB 00:00 2026-02-25T08:12:40.364 INFO:teuthology.orchestra.run.trial158.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 725 kB/s | 182 kB 00:00 2026-02-25T08:12:40.696 INFO:teuthology.orchestra.run.trial158.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 800 kB/s | 266 kB 00:00 2026-02-25T08:12:40.783 INFO:teuthology.orchestra.run.trial158.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 341 kB/s | 30 kB 00:00 2026-02-25T08:12:43.650 INFO:teuthology.orchestra.run.trial158.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 1.0 MB/s | 3.0 MB 00:02 2026-02-25T08:12:43.736 INFO:teuthology.orchestra.run.trial158.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 172 kB/s | 15 kB 00:00 2026-02-25T08:12:43.824 INFO:teuthology.orchestra.run.trial158.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 2.7 MB/s | 246 kB 00:00 2026-02-25T08:12:43.912 INFO:teuthology.orchestra.run.trial158.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 2.1 MB/s | 188 kB 00:00 2026-02-25T08:12:43.999 INFO:teuthology.orchestra.run.trial158.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 488 kB/s | 42 kB 00:00 2026-02-25T08:12:44.424 INFO:teuthology.orchestra.run.trial158.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 12 MB/s | 5.3 MB 00:00 2026-02-25T08:12:44.852 INFO:teuthology.orchestra.run.trial158.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2026-02-25T08:12:44.941 INFO:teuthology.orchestra.run.trial158.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 2.7 MB/s | 244 kB 00:00 2026-02-25T08:12:45.029 INFO:teuthology.orchestra.run.trial158.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 2.8 MB/s | 249 kB 00:00 2026-02-25T08:12:45.115 INFO:teuthology.orchestra.run.trial158.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 549 kB/s | 48 kB 00:00 2026-02-25T08:12:45.203 INFO:teuthology.orchestra.run.trial158.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 2.0 MB/s | 177 kB 00:00 2026-02-25T08:12:45.296 INFO:teuthology.orchestra.run.trial158.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2026-02-25T08:12:45.382 INFO:teuthology.orchestra.run.trial158.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 401 kB/s | 35 kB 00:00 2026-02-25T08:12:45.812 INFO:teuthology.orchestra.run.trial158.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 14 MB/s | 6.1 MB 00:00 2026-02-25T08:12:45.901 INFO:teuthology.orchestra.run.trial158.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 4.9 MB/s | 442 kB 00:00 2026-02-25T08:12:45.988 INFO:teuthology.orchestra.run.trial158.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 886 kB/s | 77 kB 00:00 2026-02-25T08:12:46.076 INFO:teuthology.orchestra.run.trial158.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 3.0 MB/s | 267 kB 00:00 2026-02-25T08:12:46.163 INFO:teuthology.orchestra.run.trial158.stdout:(48/96): python3-pyasn1-0.4.8-7.el9.noarch.rpm 1.8 MB/s | 157 kB 00:00 2026-02-25T08:12:46.251 INFO:teuthology.orchestra.run.trial158.stdout:(49/96): python3-pyasn1-modules-0.4.8-7.el9.noa 3.1 MB/s | 277 kB 00:00 2026-02-25T08:12:46.339 INFO:teuthology.orchestra.run.trial158.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 615 kB/s | 54 kB 00:00 2026-02-25T08:12:47.694 INFO:teuthology.orchestra.run.trial158.stdout:(51/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 14 MB/s | 19 MB 00:01 2026-02-25T08:12:47.781 INFO:teuthology.orchestra.run.trial158.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 474 kB/s | 42 kB 00:00 2026-02-25T08:12:47.869 INFO:teuthology.orchestra.run.trial158.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 2.1 MB/s | 189 kB 00:00 2026-02-25T08:12:47.956 INFO:teuthology.orchestra.run.trial158.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 1.0 MB/s | 90 kB 00:00 2026-02-25T08:12:48.042 INFO:teuthology.orchestra.run.trial158.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 259 kB/s | 22 kB 00:00 2026-02-25T08:12:48.132 INFO:teuthology.orchestra.run.trial158.stdout:(56/96): protobuf-compiler-3.14.0-17.el9.x86_64 9.3 MB/s | 862 kB 00:00 2026-02-25T08:12:48.704 INFO:teuthology.orchestra.run.trial158.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 964 kB/s | 551 kB 00:00 2026-02-25T08:12:48.772 INFO:teuthology.orchestra.run.trial158.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 286 kB/s | 19 kB 00:00 2026-02-25T08:12:48.852 INFO:teuthology.orchestra.run.trial158.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 1.9 MB/s | 151 kB 00:00 2026-02-25T08:12:48.966 INFO:teuthology.orchestra.run.trial158.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 4.7 MB/s | 548 kB 00:00 2026-02-25T08:12:49.034 INFO:teuthology.orchestra.run.trial158.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 430 kB/s | 29 kB 00:00 2026-02-25T08:12:49.104 INFO:teuthology.orchestra.run.trial158.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 852 kB/s | 60 kB 00:00 2026-02-25T08:12:49.175 INFO:teuthology.orchestra.run.trial158.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 624 kB/s | 43 kB 00:00 2026-02-25T08:12:49.243 INFO:teuthology.orchestra.run.trial158.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 467 kB/s | 32 kB 00:00 2026-02-25T08:12:49.311 INFO:teuthology.orchestra.run.trial158.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 209 kB/s | 14 kB 00:00 2026-02-25T08:12:49.390 INFO:teuthology.orchestra.run.trial158.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 2.1 MB/s | 173 kB 00:00 2026-02-25T08:12:49.482 INFO:teuthology.orchestra.run.trial158.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 3.8 MB/s | 358 kB 00:00 2026-02-25T08:12:49.566 INFO:teuthology.orchestra.run.trial158.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 2.9 MB/s | 254 kB 00:00 2026-02-25T08:12:49.821 INFO:teuthology.orchestra.run.trial158.stdout:(69/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.0 MB/s | 2.0 MB 00:00 2026-02-25T08:12:50.059 INFO:teuthology.orchestra.run.trial158.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 605 kB/s | 144 kB 00:00 2026-02-25T08:12:50.131 INFO:teuthology.orchestra.run.trial158.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 1.9 MB/s | 139 kB 00:00 2026-02-25T08:12:50.212 INFO:teuthology.orchestra.run.trial158.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 692 kB/s | 56 kB 00:00 2026-02-25T08:12:50.289 INFO:teuthology.orchestra.run.trial158.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 138 kB/s | 11 kB 00:00 2026-02-25T08:12:50.413 INFO:teuthology.orchestra.run.trial158.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 143 kB/s | 18 kB 00:00 2026-02-25T08:12:50.501 INFO:teuthology.orchestra.run.trial158.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 264 kB/s | 23 kB 00:00 2026-02-25T08:12:50.568 INFO:teuthology.orchestra.run.trial158.stdout:(76/96): python3-jaraco-context-6.0.1-3.el9.noa 290 kB/s | 20 kB 00:00 2026-02-25T08:12:50.637 INFO:teuthology.orchestra.run.trial158.stdout:(77/96): python3-jaraco-functools-3.5.0-2.el9.n 284 kB/s | 19 kB 00:00 2026-02-25T08:12:50.711 INFO:teuthology.orchestra.run.trial158.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 357 kB/s | 26 kB 00:00 2026-02-25T08:12:50.813 INFO:teuthology.orchestra.run.trial158.stdout:(79/96): python3-kubernetes-26.1.0-3.el9.noarch 10 MB/s | 1.0 MB 00:00 2026-02-25T08:12:50.892 INFO:teuthology.orchestra.run.trial158.stdout:(80/96): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2026-02-25T08:12:50.977 INFO:teuthology.orchestra.run.trial158.stdout:(81/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 1.0 MB/s | 86 kB 00:00 2026-02-25T08:12:51.078 INFO:teuthology.orchestra.run.trial158.stdout:(82/96): python3-natsort-7.1.1-5.el9.noarch.rpm 572 kB/s | 58 kB 00:00 2026-02-25T08:12:51.214 INFO:teuthology.orchestra.run.trial158.stdout:(83/96): python3-portend-3.1.0-2.el9.noarch.rpm 121 kB/s | 16 kB 00:00 2026-02-25T08:12:51.348 INFO:teuthology.orchestra.run.trial158.stdout:(84/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 672 kB/s | 90 kB 00:00 2026-02-25T08:12:51.438 INFO:teuthology.orchestra.run.trial158.stdout:(85/96): python3-repoze-lru-0.7-16.el9.noarch.r 343 kB/s | 31 kB 00:00 2026-02-25T08:12:51.524 INFO:teuthology.orchestra.run.trial158.stdout:(86/96): python3-routes-2.5.1-5.el9.noarch.rpm 2.1 MB/s | 188 kB 00:00 2026-02-25T08:12:51.593 INFO:teuthology.orchestra.run.trial158.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 857 kB/s | 59 kB 00:00 2026-02-25T08:12:51.664 INFO:teuthology.orchestra.run.trial158.stdout:(88/96): python3-saml-1.16.0-1.el9.noarch.rpm 1.7 MB/s | 125 kB 00:00 2026-02-25T08:12:51.734 INFO:teuthology.orchestra.run.trial158.stdout:(89/96): python3-tempora-5.0.0-2.el9.noarch.rpm 506 kB/s | 36 kB 00:00 2026-02-25T08:12:51.805 INFO:teuthology.orchestra.run.trial158.stdout:(90/96): python3-typing-extensions-4.15.0-1.el9 1.2 MB/s | 86 kB 00:00 2026-02-25T08:12:51.886 INFO:teuthology.orchestra.run.trial158.stdout:(91/96): python3-websocket-client-1.2.3-2.el9.n 1.1 MB/s | 90 kB 00:00 2026-02-25T08:12:51.962 INFO:teuthology.orchestra.run.trial158.stdout:(92/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 639 kB/s | 48 kB 00:00 2026-02-25T08:12:52.034 INFO:teuthology.orchestra.run.trial158.stdout:(93/96): python3-xmltodict-0.12.0-15.el9.noarch 309 kB/s | 22 kB 00:00 2026-02-25T08:12:52.104 INFO:teuthology.orchestra.run.trial158.stdout:(94/96): python3-zc-lockfile-2.0-10.el9.noarch. 282 kB/s | 20 kB 00:00 2026-02-25T08:14:03.923 INFO:teuthology.orchestra.run.trial006.stdout:(96/96): ceph-osd-crimson-20.3.0-5338.g6aeb6975 450 kB/s | 88 MB 03:19 2026-02-25T08:14:03.926 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:14:03.926 INFO:teuthology.orchestra.run.trial006.stdout:Total 1.1 MB/s | 226 MB 03:22 2026-02-25T08:14:04.304 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:14:04.343 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:14:04.343 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:14:05.144 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:14:05.144 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:14:05.975 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:14:05.977 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-25T08:14:05.981 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-25T08:14:06.002 INFO:teuthology.orchestra.run.trial006.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-25T08:14:06.036 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 4/96 2026-02-25T08:14:06.043 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-25T08:14:06.045 INFO:teuthology.orchestra.run.trial006.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-25T08:14:06.049 INFO:teuthology.orchestra.run.trial006.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-25T08:14:06.073 INFO:teuthology.orchestra.run.trial006.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-25T08:14:06.086 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-25T08:14:06.089 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-25T08:14:06.095 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-25T08:14:06.097 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-25T08:14:06.108 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-25T08:14:06.114 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-25T08:14:06.138 INFO:teuthology.orchestra.run.trial006.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-25T08:14:06.142 INFO:teuthology.orchestra.run.trial006.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-25T08:14:06.193 INFO:teuthology.orchestra.run.trial006.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-25T08:14:06.206 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 18/96 2026-02-25T08:14:06.208 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-25T08:14:06.213 INFO:teuthology.orchestra.run.trial006.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-25T08:14:06.218 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-25T08:14:06.223 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-25T08:14:06.226 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-25T08:14:06.241 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-25T08:14:06.245 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-25T08:14:06.250 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-25T08:14:06.261 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-25T08:14:06.270 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-25T08:14:06.277 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-25T08:14:06.296 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-25T08:14:06.308 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-25T08:14:06.323 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-25T08:14:06.328 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-25T08:14:06.336 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-25T08:14:06.376 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-25T08:14:06.704 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-25T08:14:06.714 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-25T08:14:06.718 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-25T08:14:06.724 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-25T08:14:06.726 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-25T08:14:06.753 INFO:teuthology.orchestra.run.trial006.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-25T08:14:06.797 INFO:teuthology.orchestra.run.trial006.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-25T08:14:06.806 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-25T08:14:06.812 INFO:teuthology.orchestra.run.trial006.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-25T08:14:06.815 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-25T08:14:06.821 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-25T08:14:06.824 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-25T08:14:06.831 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-25T08:14:06.835 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-25T08:14:06.863 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-25T08:14:06.874 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-25T08:14:06.880 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-25T08:14:06.887 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-25T08:14:06.922 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-25T08:14:07.148 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-25T08:14:07.172 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-25T08:14:07.176 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-25T08:14:07.235 INFO:teuthology.orchestra.run.trial006.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-25T08:14:07.237 INFO:teuthology.orchestra.run.trial006.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-25T08:14:07.259 INFO:teuthology.orchestra.run.trial006.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-25T08:14:07.588 INFO:teuthology.orchestra.run.trial006.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-25T08:14:07.662 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-25T08:14:08.339 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-25T08:14:08.355 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-25T08:14:08.365 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-25T08:14:08.372 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-25T08:14:08.386 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-25T08:14:08.401 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-25T08:14:08.478 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-25T08:14:08.486 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-25T08:14:08.510 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-25T08:14:08.540 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-25T08:14:08.591 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-25T08:14:08.597 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-25T08:14:08.601 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-25T08:14:08.602 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-25T08:14:08.620 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:14:08.621 INFO:teuthology.orchestra.run.trial006.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T08:14:08.621 INFO:teuthology.orchestra.run.trial006.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T08:14:08.621 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:08.630 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:14:08.655 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:14:08.655 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T08:14:08.655 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:08.690 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-25T08:14:09.280 INFO:teuthology.orchestra.run.trial006.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-25T08:14:14.673 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 80/96 2026-02-25T08:14:14.737 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 81/96 2026-02-25T08:14:14.740 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 82/96 2026-02-25T08:14:14.752 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 82/96 2026-02-25T08:14:14.752 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:14:14.753 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T08:14:14.753 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:14.763 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 83/96 2026-02-25T08:14:14.786 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 83/96 2026-02-25T08:14:14.786 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:14:14.786 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T08:14:14.786 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T08:14:14.786 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T08:14:14.786 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:14.845 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 84/96 2026-02-25T08:14:14.846 INFO:teuthology.orchestra.run.trial006.stdout: Installing : cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 84/96 2026-02-25T08:14:14.856 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9 85/96 2026-02-25T08:14:14.879 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el 86/96 2026-02-25T08:14:14.881 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 87/96 2026-02-25T08:14:17.784 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 87/96 2026-02-25T08:14:17.791 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 88/96 2026-02-25T08:14:18.242 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 88/96 2026-02-25T08:14:18.243 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 89/96 2026-02-25T08:14:18.256 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 89/96 2026-02-25T08:14:18.257 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 90/96 2026-02-25T08:14:18.318 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 90/96 2026-02-25T08:14:18.357 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9. 91/96 2026-02-25T08:14:18.359 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 92/96 2026-02-25T08:14:18.379 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 92/96 2026-02-25T08:14:18.379 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:14:18.379 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T08:14:18.379 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T08:14:18.379 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T08:14:18.379 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:18.391 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 93/96 2026-02-25T08:14:18.606 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 93/96 2026-02-25T08:14:18.607 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 94/96 2026-02-25T08:14:18.628 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 94/96 2026-02-25T08:14:18.628 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:14:18.628 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T08:14:18.628 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T08:14:18.629 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T08:14:18.629 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:18.710 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 95/96 2026-02-25T08:14:18.730 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 95/96 2026-02-25T08:14:18.730 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:14:18.731 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T08:14:18.731 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T08:14:18.731 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T08:14:18.731 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:18.731 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 96/96 2026-02-25T08:14:18.737 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 96/96 2026-02-25T08:14:18.761 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 96/96 2026-02-25T08:14:20.161 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 96/96 2026-02-25T08:14:20.161 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/96 2026-02-25T08:14:20.161 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/96 2026-02-25T08:14:20.161 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 4/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 5/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 6/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 7/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el 8/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 9/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 10/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 11/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 12/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9. 13/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 14/96 2026-02-25T08:14:20.162 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9 15/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 16/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 17/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-25T08:14:20.163 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-25T08:14:20.164 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 50/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 51/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-25T08:14:20.165 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-25T08:14:20.166 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-25T08:14:20.167 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-25T08:14:20.168 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-25T08:14:20.169 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-25T08:14:20.229 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-25T08:14:20.229 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:20.229 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:14:20.230 INFO:teuthology.orchestra.run.trial006.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T08:14:20.230 INFO:teuthology.orchestra.run.trial006.stdout: ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:20.230 INFO:teuthology.orchestra.run.trial006.stdout: ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:14:20.230 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:20.230 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:20.230 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:14:20.230 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:14:20.230 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:14:20.230 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:14:20.230 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:14:20.230 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T08:14:20.231 INFO:teuthology.orchestra.run.trial006.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T08:14:20.232 INFO:teuthology.orchestra.run.trial006.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T08:14:20.233 INFO:teuthology.orchestra.run.trial006.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T08:14:20.234 INFO:teuthology.orchestra.run.trial006.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T08:14:20.235 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T08:14:20.236 INFO:teuthology.orchestra.run.trial006.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T08:14:20.237 INFO:teuthology.orchestra.run.trial006.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T08:14:20.237 INFO:teuthology.orchestra.run.trial006.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T08:14:20.237 INFO:teuthology.orchestra.run.trial006.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-25T08:14:20.237 INFO:teuthology.orchestra.run.trial006.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T08:14:20.237 INFO:teuthology.orchestra.run.trial006.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T08:14:20.237 INFO:teuthology.orchestra.run.trial006.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T08:14:20.237 INFO:teuthology.orchestra.run.trial006.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-25T08:14:20.237 INFO:teuthology.orchestra.run.trial006.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-25T08:14:20.237 INFO:teuthology.orchestra.run.trial006.stdout: zip-3.0-35.el9.x86_64 2026-02-25T08:14:20.237 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:20.237 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:20.299 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-base 2026-02-25T08:14:20.624 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:27 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:20.690 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:20.708 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:20.709 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:20.709 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:20.731 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install cephadm 2026-02-25T08:14:21.047 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:28 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:21.112 INFO:teuthology.orchestra.run.trial006.stdout:Package cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:14:21.131 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:21.132 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:21.132 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:21.154 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T08:14:21.473 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:28 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:21.561 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:21.561 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:21.562 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-25T08:14:21.562 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:21.562 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:14:21.562 INFO:teuthology.orchestra.run.trial006.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 149 k 2026-02-25T08:14:21.562 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:21.562 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:14:21.562 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:21.562 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-25T08:14:21.562 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:21.562 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 149 k 2026-02-25T08:14:21.563 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 455 k 2026-02-25T08:14:21.563 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:14:22.561 INFO:teuthology.orchestra.run.trial006.stdout:ceph-immutable-object-cache-20.3.0-5338.g6aeb69 149 kB/s | 149 kB 00:00 2026-02-25T08:14:22.562 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:14:22.562 INFO:teuthology.orchestra.run.trial006.stdout:Total 149 kB/s | 149 kB 00:01 2026-02-25T08:14:22.562 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:14:22.567 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:14:22.567 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:14:22.598 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:14:22.598 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:14:22.654 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:14:22.655 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:14:22.677 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:14:22.677 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:14:22.678 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T08:14:22.678 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:22.968 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:14:22.968 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:22.968 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:14:22.969 INFO:teuthology.orchestra.run.trial006.stdout: ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:22.969 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:22.969 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:22.996 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr 2026-02-25T08:14:23.317 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:30 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:23.383 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:23.401 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:23.402 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:23.402 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:23.424 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T08:14:23.741 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:30 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:23.805 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:14:23.824 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:23.825 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:23.825 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:23.846 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T08:14:24.163 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:31 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:24.228 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:14:24.247 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:24.247 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:24.247 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:24.268 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-rook 2026-02-25T08:14:24.582 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:31 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:24.647 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:14:24.666 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:24.667 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:24.667 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:24.688 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T08:14:25.004 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:32 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:25.070 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:14:25.088 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:25.089 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:25.089 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:25.111 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-osd 2026-02-25T08:14:25.429 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:32 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:25.494 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:25.513 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:25.514 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:25.514 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:25.535 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-osd-classic 2026-02-25T08:14:25.848 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:32 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:25.914 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:25.932 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:25.933 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:25.933 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:25.953 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-fuse 2026-02-25T08:14:26.270 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:33 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:26.354 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:26.355 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:26.355 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-25T08:14:26.355 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:26.355 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:14:26.355 INFO:teuthology.orchestra.run.trial006.stdout: ceph-fuse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 914 k 2026-02-25T08:14:26.355 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:26.355 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:14:26.356 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:26.356 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-25T08:14:26.356 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:26.356 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 914 k 2026-02-25T08:14:26.356 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 2.9 M 2026-02-25T08:14:26.356 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:14:30.233 INFO:teuthology.orchestra.run.trial006.stdout:ceph-fuse-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 236 kB/s | 914 kB 00:03 2026-02-25T08:14:30.233 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:14:30.233 INFO:teuthology.orchestra.run.trial006.stdout:Total 236 kB/s | 914 kB 00:03 2026-02-25T08:14:30.233 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:14:30.238 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:14:30.238 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:14:30.271 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:14:30.272 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:14:30.363 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:14:30.381 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:30.645 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:30.707 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:30.707 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:30.707 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:14:30.707 INFO:teuthology.orchestra.run.trial006.stdout: ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:30.708 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:30.708 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:30.737 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-volume 2026-02-25T08:14:31.056 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:38 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:31.121 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:14:31.140 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:31.141 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:31.141 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:31.163 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librados-devel 2026-02-25T08:14:31.480 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:38 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:31.564 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:31.565 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:31.565 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-25T08:14:31.565 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:31.565 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:14:31.566 INFO:teuthology.orchestra.run.trial006.stdout: librados-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 124 k 2026-02-25T08:14:31.566 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:31.566 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:14:31.566 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:31.566 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-25T08:14:31.566 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:31.566 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 124 k 2026-02-25T08:14:31.566 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 434 k 2026-02-25T08:14:31.566 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:14:32.133 INFO:teuthology.orchestra.run.trial006.stdout:librados-devel-20.3.0-5338.g6aeb6975.el9.x86_64 218 kB/s | 124 kB 00:00 2026-02-25T08:14:32.134 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:14:32.135 INFO:teuthology.orchestra.run.trial006.stdout:Total 217 kB/s | 124 kB 00:00 2026-02-25T08:14:32.135 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:14:32.139 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:14:32.139 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:14:32.170 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:14:32.170 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:14:32.225 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:14:32.243 INFO:teuthology.orchestra.run.trial006.stdout: Installing : librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:32.333 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:32.398 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:32.398 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:32.398 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:14:32.398 INFO:teuthology.orchestra.run.trial006.stdout: librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:32.398 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:32.398 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:32.426 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install libcephfs2 2026-02-25T08:14:32.743 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:39 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:32.808 INFO:teuthology.orchestra.run.trial006.stdout:Package libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:32.826 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:32.827 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:32.827 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:32.849 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install libcephfs-devel 2026-02-25T08:14:33.168 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:40 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:33.252 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:33.253 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:33.253 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-25T08:14:33.254 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:33.254 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:14:33.254 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 37 k 2026-02-25T08:14:33.254 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:33.254 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:14:33.254 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:33.254 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-25T08:14:33.254 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:33.254 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 37 k 2026-02-25T08:14:33.254 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 171 k 2026-02-25T08:14:33.255 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:14:33.779 INFO:teuthology.orchestra.run.trial006.stdout:libcephfs-devel-20.3.0-5338.g6aeb6975.el9.x86_6 71 kB/s | 37 kB 00:00 2026-02-25T08:14:33.781 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:14:33.781 INFO:teuthology.orchestra.run.trial006.stdout:Total 71 kB/s | 37 kB 00:00 2026-02-25T08:14:33.781 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:14:33.783 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:14:33.784 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:14:33.786 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:14:33.786 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:14:33.808 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:14:33.826 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:33.891 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:33.953 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:33.953 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:33.953 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:14:33.953 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:33.953 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:33.953 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:33.980 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librados2 2026-02-25T08:14:34.299 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:41 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:34.365 INFO:teuthology.orchestra.run.trial006.stdout:Package librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:34.383 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:34.384 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:34.384 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:34.406 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librbd1 2026-02-25T08:14:34.725 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:41 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:34.793 INFO:teuthology.orchestra.run.trial006.stdout:Package librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:34.811 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:34.811 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:34.811 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:34.832 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rados 2026-02-25T08:14:35.149 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:42 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:35.214 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:35.231 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:35.232 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:35.232 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:35.253 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rgw 2026-02-25T08:14:35.568 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:42 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:35.633 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:35.651 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:35.651 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:35.652 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:35.673 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-cephfs 2026-02-25T08:14:35.988 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:42 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:36.053 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:36.070 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:36.071 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:36.071 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:36.092 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rbd 2026-02-25T08:14:36.408 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:43 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:36.473 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:36.491 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:36.492 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:36.492 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:36.513 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-fuse 2026-02-25T08:14:36.828 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:43 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:36.911 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:36.912 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:36.912 INFO:teuthology.orchestra.run.trial006.stdout: Package Architecture Version Repository Size 2026-02-25T08:14:36.912 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:36.912 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:14:36.912 INFO:teuthology.orchestra.run.trial006.stdout: rbd-fuse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 86 k 2026-02-25T08:14:36.912 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:36.912 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:14:36.912 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:36.912 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-25T08:14:36.912 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:36.912 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 86 k 2026-02-25T08:14:36.913 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 236 k 2026-02-25T08:14:36.913 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:14:37.482 INFO:teuthology.orchestra.run.trial006.stdout:rbd-fuse-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 151 kB/s | 86 kB 00:00 2026-02-25T08:14:37.482 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:14:37.482 INFO:teuthology.orchestra.run.trial006.stdout:Total 151 kB/s | 86 kB 00:00 2026-02-25T08:14:37.482 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:14:37.486 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:14:37.486 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:14:37.517 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:14:37.517 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:14:37.569 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:14:37.587 INFO:teuthology.orchestra.run.trial006.stdout: Installing : rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:37.674 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:37.738 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:37.738 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:37.738 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:14:37.738 INFO:teuthology.orchestra.run.trial006.stdout: rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:37.738 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:37.738 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:37.766 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-mirror 2026-02-25T08:14:38.089 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:45 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:38.172 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:38.173 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:38.173 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-25T08:14:38.173 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:38.173 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:14:38.173 INFO:teuthology.orchestra.run.trial006.stdout: rbd-mirror x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.6 M 2026-02-25T08:14:38.174 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:38.174 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:14:38.174 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:38.174 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-25T08:14:38.174 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:38.174 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 2.6 M 2026-02-25T08:14:38.174 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 11 M 2026-02-25T08:14:38.174 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:14:43.210 INFO:teuthology.orchestra.run.trial006.stdout:rbd-mirror-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 527 kB/s | 2.6 MB 00:05 2026-02-25T08:14:43.211 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:14:43.211 INFO:teuthology.orchestra.run.trial006.stdout:Total 527 kB/s | 2.6 MB 00:05 2026-02-25T08:14:43.212 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:14:43.217 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:14:43.217 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:14:43.255 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:14:43.255 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:14:43.420 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:14:43.421 INFO:teuthology.orchestra.run.trial006.stdout: Installing : rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:43.444 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:43.445 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:14:43.445 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T08:14:43.445 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T08:14:43.445 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T08:14:43.445 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:43.752 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:43.752 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:43.752 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:14:43.752 INFO:teuthology.orchestra.run.trial006.stdout: rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:43.753 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:43.753 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:43.781 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-nbd 2026-02-25T08:14:44.101 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:51 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:44.184 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:44.185 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:44.185 INFO:teuthology.orchestra.run.trial006.stdout: Package Architecture Version Repository Size 2026-02-25T08:14:44.185 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:44.185 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:14:44.185 INFO:teuthology.orchestra.run.trial006.stdout: rbd-nbd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 166 k 2026-02-25T08:14:44.186 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:44.186 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:14:44.186 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:44.186 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-25T08:14:44.186 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:44.186 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 166 k 2026-02-25T08:14:44.186 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 496 k 2026-02-25T08:14:44.186 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:14:45.175 INFO:teuthology.orchestra.run.trial006.stdout:rbd-nbd-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 168 kB/s | 166 kB 00:00 2026-02-25T08:14:45.176 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:14:45.176 INFO:teuthology.orchestra.run.trial006.stdout:Total 168 kB/s | 166 kB 00:00 2026-02-25T08:14:45.176 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:14:45.181 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:14:45.181 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:14:45.212 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:14:45.212 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:14:45.268 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:14:45.286 INFO:teuthology.orchestra.run.trial006.stdout: Installing : rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:45.372 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:45.435 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:14:45.435 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:45.435 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:14:45.435 INFO:teuthology.orchestra.run.trial006.stdout: rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:14:45.435 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:45.436 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:45.464 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-osd-crimson 2026-02-25T08:14:45.782 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:52 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:45.848 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:14:45.865 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:45.866 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:14:45.866 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:14:45.888 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-25T08:14:46.207 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:07:53 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:14:46.293 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:14:46.294 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:46.294 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-25T08:14:46.294 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:46.294 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-25T08:14:46.294 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 433 M 2026-02-25T08:14:46.294 INFO:teuthology.orchestra.run.trial006.stdout:Installing weak dependencies: 2026-02-25T08:14:46.294 INFO:teuthology.orchestra.run.trial006.stdout: ceph-debugsource x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 12 M 2026-02-25T08:14:46.294 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:46.294 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-25T08:14:46.295 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-25T08:14:46.295 INFO:teuthology.orchestra.run.trial006.stdout:Install 2 Packages 2026-02-25T08:14:46.295 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:14:46.295 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 445 M 2026-02-25T08:14:46.295 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 2.7 G 2026-02-25T08:14:46.295 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-25T08:14:55.938 INFO:teuthology.orchestra.run.trial158.stdout:(95/96): ceph-mgr-dashboard-20.3.0-5338.g6aeb69 279 kB/s | 51 MB 03:05 2026-02-25T08:15:24.839 INFO:teuthology.orchestra.run.trial158.stdout:(96/96): ceph-osd-crimson-20.3.0-5338.g6aeb6975 415 kB/s | 88 MB 03:36 2026-02-25T08:15:24.842 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:15:24.842 INFO:teuthology.orchestra.run.trial158.stdout:Total 1.0 MB/s | 226 MB 03:44 2026-02-25T08:15:25.208 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:15:25.245 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:15:25.245 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:15:26.051 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:15:26.052 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:15:26.885 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:15:26.888 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-25T08:15:26.891 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-25T08:15:26.912 INFO:teuthology.orchestra.run.trial158.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-25T08:15:26.945 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 4/96 2026-02-25T08:15:26.952 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-25T08:15:26.954 INFO:teuthology.orchestra.run.trial158.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-25T08:15:26.958 INFO:teuthology.orchestra.run.trial158.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-25T08:15:26.981 INFO:teuthology.orchestra.run.trial158.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-25T08:15:26.994 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-25T08:15:26.997 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-25T08:15:27.003 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-25T08:15:27.005 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-25T08:15:27.016 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-25T08:15:27.022 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-25T08:15:27.046 INFO:teuthology.orchestra.run.trial158.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-25T08:15:27.049 INFO:teuthology.orchestra.run.trial158.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-25T08:15:27.099 INFO:teuthology.orchestra.run.trial158.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-25T08:15:27.112 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 18/96 2026-02-25T08:15:27.114 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-25T08:15:27.121 INFO:teuthology.orchestra.run.trial158.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-25T08:15:27.126 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-25T08:15:27.131 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-25T08:15:27.134 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-25T08:15:27.148 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-25T08:15:27.153 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-25T08:15:27.157 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-25T08:15:27.169 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-25T08:15:27.178 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-25T08:15:27.184 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-25T08:15:27.203 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-25T08:15:27.214 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-25T08:15:27.230 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-25T08:15:27.234 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-25T08:15:27.242 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-25T08:15:27.281 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-25T08:15:27.604 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-25T08:15:27.614 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-25T08:15:27.618 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-25T08:15:27.624 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-25T08:15:27.625 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-25T08:15:27.653 INFO:teuthology.orchestra.run.trial158.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-25T08:15:27.696 INFO:teuthology.orchestra.run.trial158.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-25T08:15:27.704 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-25T08:15:27.710 INFO:teuthology.orchestra.run.trial158.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-25T08:15:27.714 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-25T08:15:27.719 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-25T08:15:27.723 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-25T08:15:27.730 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-25T08:15:27.733 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-25T08:15:27.761 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-25T08:15:27.771 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-25T08:15:27.776 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-25T08:15:27.783 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-25T08:15:27.818 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-25T08:15:28.043 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-25T08:15:28.067 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-25T08:15:28.070 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-25T08:15:28.130 INFO:teuthology.orchestra.run.trial158.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-25T08:15:28.138 INFO:teuthology.orchestra.run.trial158.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-25T08:15:28.153 INFO:teuthology.orchestra.run.trial158.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-25T08:15:28.479 INFO:teuthology.orchestra.run.trial158.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-25T08:15:28.553 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-25T08:15:29.226 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-25T08:15:29.243 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-25T08:15:29.253 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-25T08:15:29.259 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-25T08:15:29.273 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-25T08:15:29.288 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-25T08:15:29.363 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-25T08:15:29.372 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-25T08:15:29.395 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-25T08:15:29.425 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-25T08:15:29.475 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-25T08:15:29.481 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-25T08:15:29.484 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-25T08:15:29.486 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-25T08:15:29.504 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:15:29.505 INFO:teuthology.orchestra.run.trial158.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T08:15:29.505 INFO:teuthology.orchestra.run.trial158.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T08:15:29.505 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:29.514 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:15:29.539 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:15:29.539 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T08:15:29.539 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:29.575 INFO:teuthology.orchestra.run.trial158.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-25T08:15:30.163 INFO:teuthology.orchestra.run.trial158.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-25T08:15:35.587 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 80/96 2026-02-25T08:15:35.649 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 81/96 2026-02-25T08:15:35.652 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 82/96 2026-02-25T08:15:35.665 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 82/96 2026-02-25T08:15:35.665 INFO:teuthology.orchestra.run.trial158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:15:35.666 INFO:teuthology.orchestra.run.trial158.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T08:15:35.666 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:35.677 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 83/96 2026-02-25T08:15:35.701 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 83/96 2026-02-25T08:15:35.701 INFO:teuthology.orchestra.run.trial158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:15:35.701 INFO:teuthology.orchestra.run.trial158.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T08:15:35.701 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T08:15:35.701 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T08:15:35.701 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:35.756 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 84/96 2026-02-25T08:15:35.758 INFO:teuthology.orchestra.run.trial158.stdout: Installing : cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 84/96 2026-02-25T08:15:35.767 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9 85/96 2026-02-25T08:15:35.791 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el 86/96 2026-02-25T08:15:35.793 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 87/96 2026-02-25T08:15:36.381 INFO:teuthology.orchestra.run.trial006.stdout:(1/2): ceph-debugsource-20.3.0-5338.g6aeb6975.e 246 kB/s | 12 MB 00:50 2026-02-25T08:15:38.688 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 87/96 2026-02-25T08:15:38.694 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 88/96 2026-02-25T08:15:39.147 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 88/96 2026-02-25T08:15:39.148 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 89/96 2026-02-25T08:15:39.159 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 89/96 2026-02-25T08:15:39.159 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 90/96 2026-02-25T08:15:39.220 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 90/96 2026-02-25T08:15:39.260 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9. 91/96 2026-02-25T08:15:39.261 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 92/96 2026-02-25T08:15:39.281 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 92/96 2026-02-25T08:15:39.281 INFO:teuthology.orchestra.run.trial158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:15:39.281 INFO:teuthology.orchestra.run.trial158.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T08:15:39.281 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T08:15:39.281 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T08:15:39.281 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:39.293 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 93/96 2026-02-25T08:15:39.507 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 93/96 2026-02-25T08:15:39.508 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 94/96 2026-02-25T08:15:39.528 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 94/96 2026-02-25T08:15:39.528 INFO:teuthology.orchestra.run.trial158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:15:39.528 INFO:teuthology.orchestra.run.trial158.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T08:15:39.528 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T08:15:39.528 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T08:15:39.528 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:39.611 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 95/96 2026-02-25T08:15:39.631 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 95/96 2026-02-25T08:15:39.631 INFO:teuthology.orchestra.run.trial158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:15:39.631 INFO:teuthology.orchestra.run.trial158.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T08:15:39.631 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T08:15:39.631 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T08:15:39.631 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:39.632 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 96/96 2026-02-25T08:15:39.638 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 96/96 2026-02-25T08:15:39.662 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 96/96 2026-02-25T08:15:41.018 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 96/96 2026-02-25T08:15:41.018 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/96 2026-02-25T08:15:41.018 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/96 2026-02-25T08:15:41.018 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 4/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 5/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 6/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 7/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el 8/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 9/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 10/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 11/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 12/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9. 13/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 14/96 2026-02-25T08:15:41.019 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9 15/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 16/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 17/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-25T08:15:41.020 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-25T08:15:41.021 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 50/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 51/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-25T08:15:41.022 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-25T08:15:41.023 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-25T08:15:41.024 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-25T08:15:41.025 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-25T08:15:41.026 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-25T08:15:41.026 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-25T08:15:41.026 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-25T08:15:41.086 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-25T08:15:41.086 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:41.086 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:15:41.086 INFO:teuthology.orchestra.run.trial158.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T08:15:41.086 INFO:teuthology.orchestra.run.trial158.stdout: ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:41.086 INFO:teuthology.orchestra.run.trial158.stdout: ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:41.087 INFO:teuthology.orchestra.run.trial158.stdout: ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T08:15:41.088 INFO:teuthology.orchestra.run.trial158.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T08:15:41.089 INFO:teuthology.orchestra.run.trial158.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-25T08:15:41.090 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T08:15:41.091 INFO:teuthology.orchestra.run.trial158.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T08:15:41.092 INFO:teuthology.orchestra.run.trial158.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T08:15:41.093 INFO:teuthology.orchestra.run.trial158.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-25T08:15:41.094 INFO:teuthology.orchestra.run.trial158.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T08:15:41.094 INFO:teuthology.orchestra.run.trial158.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T08:15:41.094 INFO:teuthology.orchestra.run.trial158.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T08:15:41.094 INFO:teuthology.orchestra.run.trial158.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-25T08:15:41.094 INFO:teuthology.orchestra.run.trial158.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-25T08:15:41.094 INFO:teuthology.orchestra.run.trial158.stdout: zip-3.0-35.el9.x86_64 2026-02-25T08:15:41.094 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:41.094 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:41.155 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-base 2026-02-25T08:15:41.481 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:48 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:41.546 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:15:41.564 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:41.565 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:41.565 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:41.587 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install cephadm 2026-02-25T08:15:41.902 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:48 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:41.968 INFO:teuthology.orchestra.run.trial158.stdout:Package cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:15:41.986 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:41.987 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:41.987 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:42.009 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T08:15:42.326 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:49 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:42.410 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:42.411 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:42.411 INFO:teuthology.orchestra.run.trial158.stdout: Package Arch Version Repo Size 2026-02-25T08:15:42.411 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:42.412 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:15:42.412 INFO:teuthology.orchestra.run.trial158.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 149 k 2026-02-25T08:15:42.412 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:42.412 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:15:42.412 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:42.412 INFO:teuthology.orchestra.run.trial158.stdout:Install 1 Package 2026-02-25T08:15:42.412 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:42.412 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 149 k 2026-02-25T08:15:42.412 INFO:teuthology.orchestra.run.trial158.stdout:Installed size: 455 k 2026-02-25T08:15:42.412 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:15:42.994 INFO:teuthology.orchestra.run.trial158.stdout:ceph-immutable-object-cache-20.3.0-5338.g6aeb69 257 kB/s | 149 kB 00:00 2026-02-25T08:15:42.994 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:15:42.994 INFO:teuthology.orchestra.run.trial158.stdout:Total 256 kB/s | 149 kB 00:00 2026-02-25T08:15:42.995 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:15:42.999 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:15:42.999 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:15:43.030 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:15:43.030 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:15:43.085 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:15:43.086 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:15:43.109 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:15:43.109 INFO:teuthology.orchestra.run.trial158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:15:43.109 INFO:teuthology.orchestra.run.trial158.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T08:15:43.109 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:43.426 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:15:43.426 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:43.426 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:15:43.426 INFO:teuthology.orchestra.run.trial158.stdout: ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:43.426 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:43.426 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:43.453 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-mgr 2026-02-25T08:15:43.773 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:50 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:43.838 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:15:43.857 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:43.857 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:43.858 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:43.879 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T08:15:44.200 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:51 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:44.265 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:15:44.284 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:44.285 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:44.285 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:44.306 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T08:15:44.623 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:51 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:44.688 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:15:44.706 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:44.707 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:44.707 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:44.727 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-mgr-rook 2026-02-25T08:15:45.041 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:52 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:45.106 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:15:45.124 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:45.125 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:45.125 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:45.147 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T08:15:45.460 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:52 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:45.525 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:15:45.544 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:45.545 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:45.545 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:45.566 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-osd 2026-02-25T08:15:45.882 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:52 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:45.947 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:15:45.966 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:45.967 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:45.967 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:45.988 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-osd-classic 2026-02-25T08:15:46.302 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:53 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:46.367 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:15:46.385 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:46.386 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:46.386 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:46.408 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-fuse 2026-02-25T08:15:46.725 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:53 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:46.810 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:46.810 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:46.810 INFO:teuthology.orchestra.run.trial158.stdout: Package Arch Version Repo Size 2026-02-25T08:15:46.810 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:46.810 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:15:46.811 INFO:teuthology.orchestra.run.trial158.stdout: ceph-fuse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 914 k 2026-02-25T08:15:46.811 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:46.811 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:15:46.811 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:46.811 INFO:teuthology.orchestra.run.trial158.stdout:Install 1 Package 2026-02-25T08:15:46.811 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:46.811 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 914 k 2026-02-25T08:15:46.811 INFO:teuthology.orchestra.run.trial158.stdout:Installed size: 2.9 M 2026-02-25T08:15:46.811 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:15:48.502 INFO:teuthology.orchestra.run.trial158.stdout:ceph-fuse-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 540 kB/s | 914 kB 00:01 2026-02-25T08:15:48.502 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:15:48.503 INFO:teuthology.orchestra.run.trial158.stdout:Total 540 kB/s | 914 kB 00:01 2026-02-25T08:15:48.503 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:15:48.507 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:15:48.507 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:15:48.540 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:15:48.541 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:15:48.632 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:15:48.649 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:48.903 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:48.965 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:48.965 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:48.965 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:15:48.965 INFO:teuthology.orchestra.run.trial158.stdout: ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:48.965 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:48.965 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:48.992 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-volume 2026-02-25T08:15:49.308 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:56 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:49.374 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:15:49.393 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:49.393 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:49.394 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:49.415 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install librados-devel 2026-02-25T08:15:49.730 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:56 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:49.816 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:49.816 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:49.816 INFO:teuthology.orchestra.run.trial158.stdout: Package Arch Version Repo Size 2026-02-25T08:15:49.816 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:49.816 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:15:49.817 INFO:teuthology.orchestra.run.trial158.stdout: librados-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 124 k 2026-02-25T08:15:49.817 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:49.817 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:15:49.817 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:49.817 INFO:teuthology.orchestra.run.trial158.stdout:Install 1 Package 2026-02-25T08:15:49.817 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:49.817 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 124 k 2026-02-25T08:15:49.818 INFO:teuthology.orchestra.run.trial158.stdout:Installed size: 434 k 2026-02-25T08:15:49.818 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:15:50.731 INFO:teuthology.orchestra.run.trial158.stdout:librados-devel-20.3.0-5338.g6aeb6975.el9.x86_64 135 kB/s | 124 kB 00:00 2026-02-25T08:15:50.731 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:15:50.732 INFO:teuthology.orchestra.run.trial158.stdout:Total 135 kB/s | 124 kB 00:00 2026-02-25T08:15:50.732 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:15:50.734 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:15:50.734 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:15:50.766 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:15:50.766 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:15:50.820 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:15:50.837 INFO:teuthology.orchestra.run.trial158.stdout: Installing : librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:50.925 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:50.987 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:50.987 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:50.987 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:15:50.987 INFO:teuthology.orchestra.run.trial158.stdout: librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:50.987 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:50.987 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:51.015 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install libcephfs2 2026-02-25T08:15:51.336 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:58 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:51.402 INFO:teuthology.orchestra.run.trial158.stdout:Package libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:15:51.421 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:51.422 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:51.422 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:51.443 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install libcephfs-devel 2026-02-25T08:15:51.756 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:58 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:51.839 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:51.840 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:51.840 INFO:teuthology.orchestra.run.trial158.stdout: Package Arch Version Repo Size 2026-02-25T08:15:51.840 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:51.840 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:15:51.840 INFO:teuthology.orchestra.run.trial158.stdout: libcephfs-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 37 k 2026-02-25T08:15:51.840 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:51.841 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:15:51.841 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:51.841 INFO:teuthology.orchestra.run.trial158.stdout:Install 1 Package 2026-02-25T08:15:51.841 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:51.841 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 37 k 2026-02-25T08:15:51.841 INFO:teuthology.orchestra.run.trial158.stdout:Installed size: 171 k 2026-02-25T08:15:51.841 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:15:52.198 INFO:teuthology.orchestra.run.trial158.stdout:libcephfs-devel-20.3.0-5338.g6aeb6975.el9.x86_6 104 kB/s | 37 kB 00:00 2026-02-25T08:15:52.200 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:15:52.200 INFO:teuthology.orchestra.run.trial158.stdout:Total 103 kB/s | 37 kB 00:00 2026-02-25T08:15:52.200 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:15:52.203 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:15:52.203 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:15:52.205 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:15:52.205 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:15:52.228 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:15:52.245 INFO:teuthology.orchestra.run.trial158.stdout: Installing : libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:52.305 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:52.366 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:52.367 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:52.367 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:15:52.367 INFO:teuthology.orchestra.run.trial158.stdout: libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:52.367 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:52.367 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:52.393 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install librados2 2026-02-25T08:15:52.712 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:08:59 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:52.777 INFO:teuthology.orchestra.run.trial158.stdout:Package librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:15:52.795 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:52.795 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:52.795 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:52.818 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install librbd1 2026-02-25T08:15:53.138 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:09:00 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:53.203 INFO:teuthology.orchestra.run.trial158.stdout:Package librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:15:53.221 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:53.221 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:53.222 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:53.243 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install python3-rados 2026-02-25T08:15:53.555 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:09:00 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:53.620 INFO:teuthology.orchestra.run.trial158.stdout:Package python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:15:53.637 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:53.638 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:53.638 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:53.659 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install python3-rgw 2026-02-25T08:15:53.977 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:09:00 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:54.042 INFO:teuthology.orchestra.run.trial158.stdout:Package python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:15:54.059 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:54.060 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:54.060 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:54.082 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install python3-cephfs 2026-02-25T08:15:54.395 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:09:01 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:54.460 INFO:teuthology.orchestra.run.trial158.stdout:Package python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:15:54.478 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:54.478 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:54.479 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:54.500 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install python3-rbd 2026-02-25T08:15:54.819 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:09:01 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:54.884 INFO:teuthology.orchestra.run.trial158.stdout:Package python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:15:54.902 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:54.902 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:15:54.902 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:54.923 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install rbd-fuse 2026-02-25T08:15:55.240 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:09:02 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:55.323 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:55.324 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:55.324 INFO:teuthology.orchestra.run.trial158.stdout: Package Architecture Version Repository Size 2026-02-25T08:15:55.324 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:55.324 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:15:55.325 INFO:teuthology.orchestra.run.trial158.stdout: rbd-fuse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 86 k 2026-02-25T08:15:55.325 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:55.325 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:15:55.325 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:55.325 INFO:teuthology.orchestra.run.trial158.stdout:Install 1 Package 2026-02-25T08:15:55.325 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:55.325 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 86 k 2026-02-25T08:15:55.325 INFO:teuthology.orchestra.run.trial158.stdout:Installed size: 236 k 2026-02-25T08:15:55.325 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:15:55.960 INFO:teuthology.orchestra.run.trial158.stdout:rbd-fuse-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 136 kB/s | 86 kB 00:00 2026-02-25T08:15:55.960 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:15:55.960 INFO:teuthology.orchestra.run.trial158.stdout:Total 136 kB/s | 86 kB 00:00 2026-02-25T08:15:55.960 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:15:55.964 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:15:55.965 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:15:55.995 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:15:55.995 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:15:56.048 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:15:56.067 INFO:teuthology.orchestra.run.trial158.stdout: Installing : rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:56.153 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:56.215 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:56.215 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:56.215 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:15:56.216 INFO:teuthology.orchestra.run.trial158.stdout: rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:56.216 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:56.216 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:56.244 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install rbd-mirror 2026-02-25T08:15:56.566 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:09:03 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:15:56.649 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:15:56.650 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:56.650 INFO:teuthology.orchestra.run.trial158.stdout: Package Arch Version Repo Size 2026-02-25T08:15:56.650 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:56.650 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:15:56.650 INFO:teuthology.orchestra.run.trial158.stdout: rbd-mirror x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.6 M 2026-02-25T08:15:56.650 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:56.650 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:15:56.650 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:15:56.651 INFO:teuthology.orchestra.run.trial158.stdout:Install 1 Package 2026-02-25T08:15:56.651 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:56.651 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 2.6 M 2026-02-25T08:15:56.651 INFO:teuthology.orchestra.run.trial158.stdout:Installed size: 11 M 2026-02-25T08:15:56.651 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:15:59.033 INFO:teuthology.orchestra.run.trial158.stdout:rbd-mirror-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 1.1 MB/s | 2.6 MB 00:02 2026-02-25T08:15:59.034 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:15:59.035 INFO:teuthology.orchestra.run.trial158.stdout:Total 1.1 MB/s | 2.6 MB 00:02 2026-02-25T08:15:59.035 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:15:59.039 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:15:59.040 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:15:59.078 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:15:59.078 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:15:59.241 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:15:59.243 INFO:teuthology.orchestra.run.trial158.stdout: Installing : rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:59.265 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:59.266 INFO:teuthology.orchestra.run.trial158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:15:59.266 INFO:teuthology.orchestra.run.trial158.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T08:15:59.266 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T08:15:59.266 INFO:teuthology.orchestra.run.trial158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T08:15:59.266 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:59.604 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:15:59.604 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:59.604 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:15:59.605 INFO:teuthology.orchestra.run.trial158.stdout: rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:59.605 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:15:59.605 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:15:59.632 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install rbd-nbd 2026-02-25T08:15:59.953 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:09:06 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:16:00.037 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:16:00.037 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:16:00.037 INFO:teuthology.orchestra.run.trial158.stdout: Package Architecture Version Repository Size 2026-02-25T08:16:00.037 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:16:00.037 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:16:00.037 INFO:teuthology.orchestra.run.trial158.stdout: rbd-nbd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 166 k 2026-02-25T08:16:00.038 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:16:00.038 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:16:00.038 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:16:00.038 INFO:teuthology.orchestra.run.trial158.stdout:Install 1 Package 2026-02-25T08:16:00.038 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:16:00.038 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 166 k 2026-02-25T08:16:00.038 INFO:teuthology.orchestra.run.trial158.stdout:Installed size: 496 k 2026-02-25T08:16:00.038 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:16:00.675 INFO:teuthology.orchestra.run.trial158.stdout:rbd-nbd-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 261 kB/s | 166 kB 00:00 2026-02-25T08:16:00.675 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:16:00.675 INFO:teuthology.orchestra.run.trial158.stdout:Total 261 kB/s | 166 kB 00:00 2026-02-25T08:16:00.676 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:16:00.680 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:16:00.680 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:16:00.711 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:16:00.711 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:16:00.767 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:16:00.784 INFO:teuthology.orchestra.run.trial158.stdout: Installing : rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:16:00.871 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:16:00.934 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:16:00.934 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:16:00.934 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:16:00.934 INFO:teuthology.orchestra.run.trial158.stdout: rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:16:00.935 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:16:00.935 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:16:00.962 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-osd-crimson 2026-02-25T08:16:01.281 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:09:08 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:16:01.346 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:16:01.364 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:16:01.365 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:16:01.365 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:16:01.386 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-25T08:16:01.704 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:09:08 ago on Wed 25 Feb 2026 08:06:53 AM UTC. 2026-02-25T08:16:01.788 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:16:01.788 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:16:01.789 INFO:teuthology.orchestra.run.trial158.stdout: Package Arch Version Repo Size 2026-02-25T08:16:01.789 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:16:01.789 INFO:teuthology.orchestra.run.trial158.stdout:Installing: 2026-02-25T08:16:01.789 INFO:teuthology.orchestra.run.trial158.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 433 M 2026-02-25T08:16:01.789 INFO:teuthology.orchestra.run.trial158.stdout:Installing weak dependencies: 2026-02-25T08:16:01.789 INFO:teuthology.orchestra.run.trial158.stdout: ceph-debugsource x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 12 M 2026-02-25T08:16:01.789 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:16:01.789 INFO:teuthology.orchestra.run.trial158.stdout:Transaction Summary 2026-02-25T08:16:01.789 INFO:teuthology.orchestra.run.trial158.stdout:================================================================================ 2026-02-25T08:16:01.789 INFO:teuthology.orchestra.run.trial158.stdout:Install 2 Packages 2026-02-25T08:16:01.790 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:16:01.790 INFO:teuthology.orchestra.run.trial158.stdout:Total download size: 445 M 2026-02-25T08:16:01.790 INFO:teuthology.orchestra.run.trial158.stdout:Installed size: 2.7 G 2026-02-25T08:16:01.790 INFO:teuthology.orchestra.run.trial158.stdout:Downloading Packages: 2026-02-25T08:16:35.944 INFO:teuthology.orchestra.run.trial158.stdout:(1/2): ceph-debugsource-20.3.0-5338.g6aeb6975.e 361 kB/s | 12 MB 00:34 2026-02-25T08:37:57.187 INFO:teuthology.orchestra.run.trial006.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5338.g 318 kB/s | 433 MB 23:10 2026-02-25T08:37:57.188 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:37:57.188 INFO:teuthology.orchestra.run.trial006.stdout:Total 327 kB/s | 445 MB 23:10 2026-02-25T08:37:57.188 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-25T08:37:57.194 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-25T08:37:57.195 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-25T08:37:58.573 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-25T08:37:58.573 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-25T08:38:02.184 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-25T08:38:24.861 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/2 2026-02-25T08:38:24.879 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:38:25.712 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:38:25.712 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/2 2026-02-25T08:38:25.774 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:38:25.774 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:38:25.774 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-25T08:38:25.775 INFO:teuthology.orchestra.run.trial006.stdout: ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:38:25.775 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:38:25.775 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:38:25.775 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:38:25.848 DEBUG:teuthology.parallel:result is None 2026-02-25T08:38:55.790 INFO:teuthology.orchestra.run.trial158.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5338.g 322 kB/s | 433 MB 22:54 2026-02-25T08:38:55.791 INFO:teuthology.orchestra.run.trial158.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:38:55.791 INFO:teuthology.orchestra.run.trial158.stdout:Total 331 kB/s | 445 MB 22:54 2026-02-25T08:38:55.792 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction check 2026-02-25T08:38:55.797 INFO:teuthology.orchestra.run.trial158.stdout:Transaction check succeeded. 2026-02-25T08:38:55.797 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction test 2026-02-25T08:38:57.173 INFO:teuthology.orchestra.run.trial158.stdout:Transaction test succeeded. 2026-02-25T08:38:57.173 INFO:teuthology.orchestra.run.trial158.stdout:Running transaction 2026-02-25T08:39:00.760 INFO:teuthology.orchestra.run.trial158.stdout: Preparing : 1/1 2026-02-25T08:39:23.378 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/2 2026-02-25T08:39:23.396 INFO:teuthology.orchestra.run.trial158.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:39:24.268 INFO:teuthology.orchestra.run.trial158.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:39:24.268 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/2 2026-02-25T08:39:24.331 INFO:teuthology.orchestra.run.trial158.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:39:24.332 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:39:24.332 INFO:teuthology.orchestra.run.trial158.stdout:Installed: 2026-02-25T08:39:24.332 INFO:teuthology.orchestra.run.trial158.stdout: ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:39:24.332 INFO:teuthology.orchestra.run.trial158.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:39:24.332 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:39:24.332 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:24.409 DEBUG:teuthology.parallel:result is None 2026-02-25T08:39:24.410 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:39:24.574 DEBUG:teuthology.orchestra.run.trial006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T08:39:24.596 INFO:teuthology.orchestra.run.trial006.stdout:20.3.0-5338.g6aeb6975.el9 2026-02-25T08:39:24.597 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5338.g6aeb6975.el9 2026-02-25T08:39:24.597 INFO:teuthology.task.install:The correct ceph version 20.3.0-5338.g6aeb6975 is installed. 2026-02-25T08:39:24.600 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:39:24.756 DEBUG:teuthology.orchestra.run.trial158:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T08:39:24.777 INFO:teuthology.orchestra.run.trial158.stdout:20.3.0-5338.g6aeb6975.el9 2026-02-25T08:39:24.777 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5338.g6aeb6975.el9 2026-02-25T08:39:24.778 INFO:teuthology.task.install:The correct ceph version 20.3.0-5338.g6aeb6975 is installed. 2026-02-25T08:39:24.780 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-25T08:39:24.781 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:39:24.781 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T08:39:24.808 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:39:24.808 DEBUG:teuthology.orchestra.run.trial158:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T08:39:24.842 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-25T08:39:24.842 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:39:24.842 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T08:39:24.873 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T08:39:24.938 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:39:24.938 DEBUG:teuthology.orchestra.run.trial158:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T08:39:24.963 DEBUG:teuthology.orchestra.run.trial158:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T08:39:25.027 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-25T08:39:25.027 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:39:25.027 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T08:39:25.053 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T08:39:25.117 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:39:25.117 DEBUG:teuthology.orchestra.run.trial158:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T08:39:25.142 DEBUG:teuthology.orchestra.run.trial158:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T08:39:25.207 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-25T08:39:25.207 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:39:25.207 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T08:39:25.233 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T08:39:25.297 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:39:25.297 DEBUG:teuthology.orchestra.run.trial158:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T08:39:25.322 DEBUG:teuthology.orchestra.run.trial158:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T08:39:25.386 INFO:teuthology.run_tasks:Running task pexec... 2026-02-25T08:39:25.404 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-25T08:39:25.404 DEBUG:teuthology.orchestra.run.trial006:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-25T08:39:25.405 DEBUG:teuthology.orchestra.run.trial158:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-25T08:39:25.408 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial006.front.sepia.ceph.com 2026-02-25T08:39:25.408 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-25T08:39:25.428 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial158.front.sepia.ceph.com 2026-02-25T08:39:25.429 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-25T08:39:25.458 DEBUG:teuthology.parallel:result is None 2026-02-25T08:39:25.478 DEBUG:teuthology.parallel:result is None 2026-02-25T08:39:25.478 INFO:teuthology.run_tasks:Running task install... 2026-02-25T08:39:25.492 DEBUG:teuthology.task.install:project ceph 2026-02-25T08:39:25.492 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}} 2026-02-25T08:39:25.492 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'} 2026-02-25T08:39:25.493 INFO:teuthology.task.install:Using flavor: debug 2026-02-25T08:39:25.501 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-25T08:39:25.501 INFO:teuthology.task.install:extra packages: [] 2026-02-25T08:39:25.502 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'tag': None, 'wait_for_package': False} 2026-02-25T08:39:25.502 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:39:25.504 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'tag': None, 'wait_for_package': False} 2026-02-25T08:39:25.504 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:39:25.694 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/ 2026-02-25T08:39:25.694 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5338.g6aeb6975 2026-02-25T08:39:25.705 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/ 2026-02-25T08:39:25.705 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5338.g6aeb6975 2026-02-25T08:39:26.068 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T08:39:26.068 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:39:26.068 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T08:39:26.093 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-25T08:39:26.094 DEBUG:teuthology.orchestra.run.trial006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T08:39:26.112 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T08:39:26.112 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:39:26.112 DEBUG:teuthology.orchestra.run.trial158:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T08:39:26.137 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-25T08:39:26.138 DEBUG:teuthology.orchestra.run.trial158:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T08:39:26.157 DEBUG:teuthology.orchestra.run.trial006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T08:39:26.202 DEBUG:teuthology.orchestra.run.trial158:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T08:39:26.222 DEBUG:teuthology.orchestra.run.trial006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T08:39:26.266 DEBUG:teuthology.orchestra.run.trial158:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T08:39:26.278 INFO:teuthology.orchestra.run.trial006.stdout:check_obsoletes = 1 2026-02-25T08:39:26.288 DEBUG:teuthology.orchestra.run.trial006:> sudo yum clean all 2026-02-25T08:39:26.322 INFO:teuthology.orchestra.run.trial158.stdout:check_obsoletes = 1 2026-02-25T08:39:26.332 DEBUG:teuthology.orchestra.run.trial158:> sudo yum clean all 2026-02-25T08:39:26.449 INFO:teuthology.orchestra.run.trial006.stdout:56 files removed 2026-02-25T08:39:26.465 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-radosgw 2026-02-25T08:39:26.492 INFO:teuthology.orchestra.run.trial158.stdout:56 files removed 2026-02-25T08:39:26.507 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-radosgw 2026-02-25T08:39:27.320 INFO:teuthology.orchestra.run.trial006.stdout:ceph packages for x86_64 134 kB/s | 94 kB 00:00 2026-02-25T08:39:27.354 INFO:teuthology.orchestra.run.trial158.stdout:ceph packages for x86_64 136 kB/s | 94 kB 00:00 2026-02-25T08:39:28.061 INFO:teuthology.orchestra.run.trial006.stdout:ceph noarch packages 34 kB/s | 24 kB 00:00 2026-02-25T08:39:28.094 INFO:teuthology.orchestra.run.trial158.stdout:ceph noarch packages 34 kB/s | 24 kB 00:00 2026-02-25T08:39:28.640 INFO:teuthology.orchestra.run.trial006.stdout:ceph source packages 3.7 kB/s | 2.1 kB 00:00 2026-02-25T08:39:28.679 INFO:teuthology.orchestra.run.trial158.stdout:ceph source packages 3.7 kB/s | 2.1 kB 00:00 2026-02-25T08:39:29.440 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.9 MB 00:00 2026-02-25T08:39:30.487 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.9 MB 00:01 2026-02-25T08:39:32.563 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-25T08:39:35.962 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 8.0 MB 00:01 2026-02-25T08:39:37.755 INFO:teuthology.orchestra.run.trial158.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 20 kB 00:01 2026-02-25T08:39:38.909 INFO:teuthology.orchestra.run.trial158.stdout:Extra Packages for Enterprise Linux 19 MB/s | 20 MB 00:01 2026-02-25T08:39:39.010 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - AppStream 3.4 MB/s | 27 MB 00:07 2026-02-25T08:39:42.074 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - CRB 13 MB/s | 8.0 MB 00:00 2026-02-25T08:39:42.131 INFO:teuthology.orchestra.run.trial158.stdout:lab-extras 1.1 MB/s | 50 kB 00:00 2026-02-25T08:39:42.973 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - Extras packages 128 kB/s | 20 kB 00:00 2026-02-25T08:39:43.314 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:43.332 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:43.332 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:43.332 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:43.357 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-test 2026-02-25T08:39:43.684 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:01 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:43.749 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:43.766 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:43.767 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:43.767 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:43.789 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph 2026-02-25T08:39:44.078 INFO:teuthology.orchestra.run.trial006.stdout:Extra Packages for Enterprise Linux 20 MB/s | 20 MB 00:01 2026-02-25T08:39:44.102 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:44.167 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:44.185 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:44.185 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:44.185 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:44.206 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-base 2026-02-25T08:39:44.524 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:44.589 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:44.606 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:44.607 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:44.607 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:44.628 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install cephadm 2026-02-25T08:39:44.949 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:45.014 INFO:teuthology.orchestra.run.trial158.stdout:Package cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:45.032 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:45.032 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:45.033 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:45.054 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T08:39:45.370 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:45.436 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:45.454 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:45.454 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:45.454 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:45.475 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-mgr 2026-02-25T08:39:45.792 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:45.858 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:45.875 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:45.876 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:45.876 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:45.898 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T08:39:46.218 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:46.283 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:46.301 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:46.302 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:46.302 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:46.324 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T08:39:46.646 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:46.710 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:46.728 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:46.728 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:46.728 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:46.749 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-mgr-rook 2026-02-25T08:39:47.064 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:47.129 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:47.147 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:47.147 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:47.147 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:47.169 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T08:39:47.296 INFO:teuthology.orchestra.run.trial006.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-25T08:39:47.489 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:47.554 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:47.571 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:47.572 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:47.572 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:47.594 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-osd 2026-02-25T08:39:47.916 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:47.981 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:47.999 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:47.999 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:47.999 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:48.021 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-osd-classic 2026-02-25T08:39:48.337 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:48.406 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:48.424 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:48.425 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:48.425 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:48.445 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-fuse 2026-02-25T08:39:48.485 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:48.503 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:48.504 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:48.504 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:48.528 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-test 2026-02-25T08:39:48.765 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:48.830 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:48.848 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:48.848 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:48.848 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:48.849 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:01 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:48.870 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install ceph-volume 2026-02-25T08:39:48.916 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:48.934 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:48.935 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:48.935 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:48.956 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph 2026-02-25T08:39:49.187 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:49.252 INFO:teuthology.orchestra.run.trial158.stdout:Package ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:49.270 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:49.271 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:49.271 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:49.271 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:49.292 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install librados-devel 2026-02-25T08:39:49.336 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:49.354 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:49.354 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:49.354 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:49.376 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-base 2026-02-25T08:39:49.608 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:49.673 INFO:teuthology.orchestra.run.trial158.stdout:Package librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:49.691 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:49.691 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:49.692 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:49.698 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:49.712 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install libcephfs2 2026-02-25T08:39:49.763 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:49.781 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:49.782 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:49.782 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:49.804 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install cephadm 2026-02-25T08:39:50.032 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:50.097 INFO:teuthology.orchestra.run.trial158.stdout:Package libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:50.115 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:50.115 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:50.116 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:50.125 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:50.137 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install libcephfs-devel 2026-02-25T08:39:50.190 INFO:teuthology.orchestra.run.trial006.stdout:Package cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:50.208 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:50.208 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:50.208 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:50.231 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T08:39:50.455 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:50.520 INFO:teuthology.orchestra.run.trial158.stdout:Package libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:50.538 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:50.538 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:50.538 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:50.550 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:50.560 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install librados2 2026-02-25T08:39:50.615 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:50.633 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:50.633 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:50.633 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:50.654 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr 2026-02-25T08:39:50.878 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:50.943 INFO:teuthology.orchestra.run.trial158.stdout:Package librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:50.960 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:50.961 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:50.961 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:50.971 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:50.982 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install librbd1 2026-02-25T08:39:51.036 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:51.054 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:51.055 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:51.055 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:51.076 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T08:39:51.301 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:51.367 INFO:teuthology.orchestra.run.trial158.stdout:Package librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:51.385 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:51.385 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:51.385 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:51.391 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:51.407 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install python3-rados 2026-02-25T08:39:51.456 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:51.474 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:51.474 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:51.475 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:51.495 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T08:39:51.729 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:51.794 INFO:teuthology.orchestra.run.trial158.stdout:Package python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:51.812 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:51.812 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:51.813 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:51.813 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:51.835 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install python3-rgw 2026-02-25T08:39:51.877 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:51.895 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:51.896 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:51.896 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:51.916 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-rook 2026-02-25T08:39:52.148 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:52.213 INFO:teuthology.orchestra.run.trial158.stdout:Package python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:52.231 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:52.231 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:52.231 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:52.233 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:52.253 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install python3-cephfs 2026-02-25T08:39:52.298 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:52.316 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:52.316 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:52.317 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:52.337 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T08:39:52.572 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:52.637 INFO:teuthology.orchestra.run.trial158.stdout:Package python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:52.648 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:52.655 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:52.655 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:52.656 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:52.677 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install python3-rbd 2026-02-25T08:39:52.714 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:52.732 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:52.732 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:52.733 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:52.754 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-osd 2026-02-25T08:39:52.995 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:53.061 INFO:teuthology.orchestra.run.trial158.stdout:Package python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:53.076 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:53.079 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:53.080 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:53.080 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:53.101 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install rbd-fuse 2026-02-25T08:39:53.142 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:53.159 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:53.160 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:53.160 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:53.182 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-osd-classic 2026-02-25T08:39:53.418 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:53.484 INFO:teuthology.orchestra.run.trial158.stdout:Package rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:53.494 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:53.502 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:53.502 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:53.502 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:53.524 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install rbd-mirror 2026-02-25T08:39:53.560 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:53.578 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:53.579 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:53.579 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:53.600 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-fuse 2026-02-25T08:39:53.839 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:53.906 INFO:teuthology.orchestra.run.trial158.stdout:Package rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:53.922 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:53.924 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:53.924 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:53.924 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:53.946 DEBUG:teuthology.orchestra.run.trial158:> sudo yum -y install rbd-nbd 2026-02-25T08:39:53.988 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:54.006 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:54.006 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:54.007 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:54.029 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-volume 2026-02-25T08:39:54.266 INFO:teuthology.orchestra.run.trial158.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 08:39:42 AM UTC. 2026-02-25T08:39:54.332 INFO:teuthology.orchestra.run.trial158.stdout:Package rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:54.348 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:54.349 INFO:teuthology.orchestra.run.trial158.stdout:Dependencies resolved. 2026-02-25T08:39:54.350 INFO:teuthology.orchestra.run.trial158.stdout:Nothing to do. 2026-02-25T08:39:54.350 INFO:teuthology.orchestra.run.trial158.stdout:Complete! 2026-02-25T08:39:54.376 DEBUG:teuthology.parallel:result is None 2026-02-25T08:39:54.414 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:39:54.431 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:54.432 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:54.432 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:54.454 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librados-devel 2026-02-25T08:39:54.771 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:54.836 INFO:teuthology.orchestra.run.trial006.stdout:Package librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:54.854 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:54.854 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:54.855 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:54.876 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install libcephfs2 2026-02-25T08:39:55.197 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:55.263 INFO:teuthology.orchestra.run.trial006.stdout:Package libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:55.281 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:55.281 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:55.281 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:55.304 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install libcephfs-devel 2026-02-25T08:39:55.622 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:55.687 INFO:teuthology.orchestra.run.trial006.stdout:Package libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:55.705 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:55.705 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:55.705 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:55.727 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librados2 2026-02-25T08:39:56.047 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:56.113 INFO:teuthology.orchestra.run.trial006.stdout:Package librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:56.130 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:56.131 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:56.131 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:56.154 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librbd1 2026-02-25T08:39:56.478 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:56.544 INFO:teuthology.orchestra.run.trial006.stdout:Package librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:56.561 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:56.562 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:56.562 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:56.585 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rados 2026-02-25T08:39:56.901 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:56.967 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:56.985 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:56.985 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:56.985 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:57.007 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rgw 2026-02-25T08:39:57.324 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:57.389 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:57.407 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:57.407 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:57.407 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:57.429 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-cephfs 2026-02-25T08:39:57.743 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:57.809 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:57.827 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:57.827 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:57.827 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:57.849 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rbd 2026-02-25T08:39:58.166 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:58.231 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:58.249 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:58.250 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:58.250 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:58.272 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-fuse 2026-02-25T08:39:58.598 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:58.663 INFO:teuthology.orchestra.run.trial006.stdout:Package rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:58.681 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:58.682 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:58.682 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:58.704 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-mirror 2026-02-25T08:39:59.020 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:59.086 INFO:teuthology.orchestra.run.trial006.stdout:Package rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:59.104 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:59.104 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:59.104 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:59.126 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-nbd 2026-02-25T08:39:59.444 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 08:39:47 AM UTC. 2026-02-25T08:39:59.509 INFO:teuthology.orchestra.run.trial006.stdout:Package rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:39:59.527 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-25T08:39:59.527 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-25T08:39:59.528 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-25T08:39:59.552 DEBUG:teuthology.parallel:result is None 2026-02-25T08:39:59.553 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:39:59.713 DEBUG:teuthology.orchestra.run.trial006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T08:39:59.735 INFO:teuthology.orchestra.run.trial006.stdout:20.3.0-5338.g6aeb6975.el9 2026-02-25T08:39:59.735 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5338.g6aeb6975.el9 2026-02-25T08:39:59.735 INFO:teuthology.task.install:The correct ceph version 20.3.0-5338.g6aeb6975 is installed. 2026-02-25T08:39:59.738 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:39:59.894 DEBUG:teuthology.orchestra.run.trial158:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T08:39:59.916 INFO:teuthology.orchestra.run.trial158.stdout:20.3.0-5338.g6aeb6975.el9 2026-02-25T08:39:59.916 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5338.g6aeb6975.el9 2026-02-25T08:39:59.916 INFO:teuthology.task.install:The correct ceph version 20.3.0-5338.g6aeb6975 is installed. 2026-02-25T08:39:59.919 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-25T08:39:59.919 INFO:teuthology.run_tasks:Running task ceph... 2026-02-25T08:40:00.052 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-25T08:40:00.052 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod 777 /var/log/ceph 2026-02-25T08:40:00.054 DEBUG:teuthology.orchestra.run.trial158:> sudo chmod 777 /var/log/ceph 2026-02-25T08:40:00.079 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-25T08:40:00.079 DEBUG:teuthology.orchestra.run.trial006:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-25T08:40:00.120 DEBUG:teuthology.orchestra.run.trial158:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-25T08:40:00.143 INFO:tasks.ceph:Creating extra log directories... 2026-02-25T08:40:00.144 DEBUG:teuthology.orchestra.run.trial006:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-25T08:40:00.184 DEBUG:teuthology.orchestra.run.trial158:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-25T08:40:00.220 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-25T08:40:00.220 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd target transaction size': 5, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'Degraded data redundancy', 'is down', 'osds down', 'pg .*? is .*?degraded.*?, acting', 'pg .*? is stuck', 'pg degraded', 'PG_BACKFILL_FULL', 'Low space hindering backfill .*? backfill_toofull', 'OSD_ROOT_DOWN', 'pgs degraded', 'pgs undersized', 'is active.*backfill_toofull.*'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-25T08:40:00.221 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69032', 'branch': 'wip-logstoremanager', 'description': 'crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{default} thrashosds-health workloads/snaps-few-objects-localized}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '69032', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-logstoremanager'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd target transaction size': 5, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'Degraded data redundancy', 'is down', 'osds down', 'pg .*? is .*?degraded.*?, acting', 'pg .*? is stuck', 'pg degraded', 'PG_BACKFILL_FULL', 'Low space hindering backfill .*? backfill_toofull', 'OSD_ROOT_DOWN', 'pgs degraded', 'pgs undersized', 'is active.*backfill_toofull.*'], 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'c32469671c6b0222f786dd36c34eb702f780a11d'}}, 'owner': 'scheduled_matan@soko04.front.sepia.ceph.com', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 869, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': 'c32469671c6b0222f786dd36c34eb702f780a11d', 'targets': {'trial006.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBY8KPfLJtrb5ih0DtRFYMLj6uAIodB5iOSbqQAMsycvZ7aFsIfsOyn49oQfJzjkN4Wu7dyyiXnf41krlwJyqIk=', 'trial158.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCz43pSNwRLgrgH+0Ke9zmdsvWRMOFwzOEozp2jrQafGic7WXD6ygD3194phBL/Wshr/5nhbtjhnbTLtSTWy62Y='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd target transaction size': 5, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'Degraded data redundancy', 'is down', 'osds down', 'pg .*? is .*?degraded.*?, acting', 'pg .*? is stuck', 'pg degraded', 'PG_BACKFILL_FULL', 'Low space hindering backfill .*? backfill_toofull', 'OSD_ROOT_DOWN', 'pgs degraded', 'pgs undersized', 'is active.*backfill_toofull.*'], 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'cluster': 'ceph'}}, {'thrashosds': {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'chance_trim_stale_osdmaps': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400}}, {'rados': {'clients': ['client.0'], 'localize_reads': True, 'objects': 50, 'op_weights': {'copy_from': 0, 'delete': 50, 'read': 100, 'rollback': 0, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-25_07:55:35', 'tube': 'trial', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699'} 2026-02-25T08:40:00.222 DEBUG:teuthology.orchestra.run.trial006:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-25T08:40:00.248 DEBUG:teuthology.orchestra.run.trial158:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-25T08:40:00.265 DEBUG:teuthology.orchestra.run.trial006:> sudo install -d -m0777 -- /var/run/ceph 2026-02-25T08:40:00.306 DEBUG:teuthology.orchestra.run.trial158:> sudo install -d -m0777 -- /var/run/ceph 2026-02-25T08:40:00.330 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:40:00.330 DEBUG:teuthology.orchestra.run.trial006:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T08:40:00.385 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T08:40:00.386 DEBUG:teuthology.orchestra.run.trial006:> stat /dev/vg_nvme/lv_1 2026-02-25T08:40:00.442 INFO:teuthology.orchestra.run.trial006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T08:40:00.443 INFO:teuthology.orchestra.run.trial006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:00.443 INFO:teuthology.orchestra.run.trial006.stdout:Device: 6h/6d Inode: 1313 Links: 1 2026-02-25T08:40:00.443 INFO:teuthology.orchestra.run.trial006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:00.443 INFO:teuthology.orchestra.run.trial006.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:00.443 INFO:teuthology.orchestra.run.trial006.stdout:Access: 2026-02-25 08:14:43.585897242 +0000 2026-02-25T08:40:00.443 INFO:teuthology.orchestra.run.trial006.stdout:Modify: 2026-02-25 08:07:13.409217563 +0000 2026-02-25T08:40:00.443 INFO:teuthology.orchestra.run.trial006.stdout:Change: 2026-02-25 08:07:13.409217563 +0000 2026-02-25T08:40:00.443 INFO:teuthology.orchestra.run.trial006.stdout: Birth: 2026-02-25 08:07:13.409217563 +0000 2026-02-25T08:40:00.444 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T08:40:00.505 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records in 2026-02-25T08:40:00.505 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records out 2026-02-25T08:40:00.505 INFO:teuthology.orchestra.run.trial006.stderr:512 bytes copied, 0.000149598 s, 3.4 MB/s 2026-02-25T08:40:00.506 DEBUG:teuthology.orchestra.run.trial006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T08:40:00.564 DEBUG:teuthology.orchestra.run.trial006:> stat /dev/vg_nvme/lv_2 2026-02-25T08:40:00.621 INFO:teuthology.orchestra.run.trial006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T08:40:00.621 INFO:teuthology.orchestra.run.trial006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:00.621 INFO:teuthology.orchestra.run.trial006.stdout:Device: 6h/6d Inode: 1304 Links: 1 2026-02-25T08:40:00.621 INFO:teuthology.orchestra.run.trial006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:00.621 INFO:teuthology.orchestra.run.trial006.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:00.621 INFO:teuthology.orchestra.run.trial006.stdout:Access: 2026-02-25 08:14:43.586897217 +0000 2026-02-25T08:40:00.621 INFO:teuthology.orchestra.run.trial006.stdout:Modify: 2026-02-25 08:07:13.408217602 +0000 2026-02-25T08:40:00.621 INFO:teuthology.orchestra.run.trial006.stdout:Change: 2026-02-25 08:07:13.408217602 +0000 2026-02-25T08:40:00.622 INFO:teuthology.orchestra.run.trial006.stdout: Birth: 2026-02-25 08:07:13.408217602 +0000 2026-02-25T08:40:00.622 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T08:40:00.684 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records in 2026-02-25T08:40:00.684 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records out 2026-02-25T08:40:00.685 INFO:teuthology.orchestra.run.trial006.stderr:512 bytes copied, 0.000154627 s, 3.3 MB/s 2026-02-25T08:40:00.686 DEBUG:teuthology.orchestra.run.trial006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T08:40:00.744 DEBUG:teuthology.orchestra.run.trial006:> stat /dev/vg_nvme/lv_3 2026-02-25T08:40:00.800 INFO:teuthology.orchestra.run.trial006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T08:40:00.800 INFO:teuthology.orchestra.run.trial006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:00.800 INFO:teuthology.orchestra.run.trial006.stdout:Device: 6h/6d Inode: 1297 Links: 1 2026-02-25T08:40:00.801 INFO:teuthology.orchestra.run.trial006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:00.801 INFO:teuthology.orchestra.run.trial006.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:00.801 INFO:teuthology.orchestra.run.trial006.stdout:Access: 2026-02-25 08:14:43.586897217 +0000 2026-02-25T08:40:00.801 INFO:teuthology.orchestra.run.trial006.stdout:Modify: 2026-02-25 08:07:13.407217641 +0000 2026-02-25T08:40:00.801 INFO:teuthology.orchestra.run.trial006.stdout:Change: 2026-02-25 08:07:13.407217641 +0000 2026-02-25T08:40:00.801 INFO:teuthology.orchestra.run.trial006.stdout: Birth: 2026-02-25 08:07:13.407217641 +0000 2026-02-25T08:40:00.801 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T08:40:00.862 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records in 2026-02-25T08:40:00.862 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records out 2026-02-25T08:40:00.863 INFO:teuthology.orchestra.run.trial006.stderr:512 bytes copied, 0.000144829 s, 3.5 MB/s 2026-02-25T08:40:00.864 DEBUG:teuthology.orchestra.run.trial006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T08:40:00.920 DEBUG:teuthology.orchestra.run.trial006:> stat /dev/vg_nvme/lv_4 2026-02-25T08:40:00.976 INFO:teuthology.orchestra.run.trial006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T08:40:00.976 INFO:teuthology.orchestra.run.trial006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:00.976 INFO:teuthology.orchestra.run.trial006.stdout:Device: 6h/6d Inode: 1293 Links: 1 2026-02-25T08:40:00.976 INFO:teuthology.orchestra.run.trial006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:00.977 INFO:teuthology.orchestra.run.trial006.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:00.977 INFO:teuthology.orchestra.run.trial006.stdout:Access: 2026-02-25 08:14:43.586897217 +0000 2026-02-25T08:40:00.977 INFO:teuthology.orchestra.run.trial006.stdout:Modify: 2026-02-25 08:07:13.407217641 +0000 2026-02-25T08:40:00.977 INFO:teuthology.orchestra.run.trial006.stdout:Change: 2026-02-25 08:07:13.407217641 +0000 2026-02-25T08:40:00.977 INFO:teuthology.orchestra.run.trial006.stdout: Birth: 2026-02-25 08:07:13.407217641 +0000 2026-02-25T08:40:00.977 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T08:40:01.038 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records in 2026-02-25T08:40:01.038 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records out 2026-02-25T08:40:01.038 INFO:teuthology.orchestra.run.trial006.stderr:512 bytes copied, 0.000142766 s, 3.6 MB/s 2026-02-25T08:40:01.039 DEBUG:teuthology.orchestra.run.trial006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T08:40:01.098 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:01.098 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:40:01.098 DEBUG:teuthology.orchestra.run.trial158:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T08:40:01.116 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T08:40:01.117 DEBUG:teuthology.orchestra.run.trial158:> stat /dev/vg_nvme/lv_1 2026-02-25T08:40:01.172 INFO:teuthology.orchestra.run.trial158.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T08:40:01.172 INFO:teuthology.orchestra.run.trial158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:01.172 INFO:teuthology.orchestra.run.trial158.stdout:Device: 6h/6d Inode: 1288 Links: 1 2026-02-25T08:40:01.172 INFO:teuthology.orchestra.run.trial158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:01.172 INFO:teuthology.orchestra.run.trial158.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:01.172 INFO:teuthology.orchestra.run.trial158.stdout:Access: 2026-02-25 08:15:59.408100150 +0000 2026-02-25T08:40:01.172 INFO:teuthology.orchestra.run.trial158.stdout:Modify: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.172 INFO:teuthology.orchestra.run.trial158.stdout:Change: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.173 INFO:teuthology.orchestra.run.trial158.stdout: Birth: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.173 DEBUG:teuthology.orchestra.run.trial158:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T08:40:01.234 INFO:teuthology.orchestra.run.trial158.stderr:1+0 records in 2026-02-25T08:40:01.234 INFO:teuthology.orchestra.run.trial158.stderr:1+0 records out 2026-02-25T08:40:01.234 INFO:teuthology.orchestra.run.trial158.stderr:512 bytes copied, 0.000142969 s, 3.6 MB/s 2026-02-25T08:40:01.236 DEBUG:teuthology.orchestra.run.trial158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T08:40:01.293 DEBUG:teuthology.orchestra.run.trial158:> stat /dev/vg_nvme/lv_2 2026-02-25T08:40:01.349 INFO:teuthology.orchestra.run.trial158.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T08:40:01.349 INFO:teuthology.orchestra.run.trial158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:01.349 INFO:teuthology.orchestra.run.trial158.stdout:Device: 6h/6d Inode: 1307 Links: 1 2026-02-25T08:40:01.349 INFO:teuthology.orchestra.run.trial158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:01.349 INFO:teuthology.orchestra.run.trial158.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:01.350 INFO:teuthology.orchestra.run.trial158.stdout:Access: 2026-02-25 08:15:59.409100154 +0000 2026-02-25T08:40:01.350 INFO:teuthology.orchestra.run.trial158.stdout:Modify: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.350 INFO:teuthology.orchestra.run.trial158.stdout:Change: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.350 INFO:teuthology.orchestra.run.trial158.stdout: Birth: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.350 DEBUG:teuthology.orchestra.run.trial158:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T08:40:01.411 INFO:teuthology.orchestra.run.trial158.stderr:1+0 records in 2026-02-25T08:40:01.411 INFO:teuthology.orchestra.run.trial158.stderr:1+0 records out 2026-02-25T08:40:01.411 INFO:teuthology.orchestra.run.trial158.stderr:512 bytes copied, 0.000161343 s, 3.2 MB/s 2026-02-25T08:40:01.412 DEBUG:teuthology.orchestra.run.trial158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T08:40:01.469 DEBUG:teuthology.orchestra.run.trial158:> stat /dev/vg_nvme/lv_3 2026-02-25T08:40:01.525 INFO:teuthology.orchestra.run.trial158.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T08:40:01.525 INFO:teuthology.orchestra.run.trial158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:01.525 INFO:teuthology.orchestra.run.trial158.stdout:Device: 6h/6d Inode: 1289 Links: 1 2026-02-25T08:40:01.525 INFO:teuthology.orchestra.run.trial158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:01.525 INFO:teuthology.orchestra.run.trial158.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:01.525 INFO:teuthology.orchestra.run.trial158.stdout:Access: 2026-02-25 08:15:59.409100154 +0000 2026-02-25T08:40:01.525 INFO:teuthology.orchestra.run.trial158.stdout:Modify: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.526 INFO:teuthology.orchestra.run.trial158.stdout:Change: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.526 INFO:teuthology.orchestra.run.trial158.stdout: Birth: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.526 DEBUG:teuthology.orchestra.run.trial158:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T08:40:01.588 INFO:teuthology.orchestra.run.trial158.stderr:1+0 records in 2026-02-25T08:40:01.588 INFO:teuthology.orchestra.run.trial158.stderr:1+0 records out 2026-02-25T08:40:01.588 INFO:teuthology.orchestra.run.trial158.stderr:512 bytes copied, 0.000150973 s, 3.4 MB/s 2026-02-25T08:40:01.590 DEBUG:teuthology.orchestra.run.trial158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T08:40:01.646 DEBUG:teuthology.orchestra.run.trial158:> stat /dev/vg_nvme/lv_4 2026-02-25T08:40:01.702 INFO:teuthology.orchestra.run.trial158.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T08:40:01.702 INFO:teuthology.orchestra.run.trial158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:01.702 INFO:teuthology.orchestra.run.trial158.stdout:Device: 6h/6d Inode: 1302 Links: 1 2026-02-25T08:40:01.702 INFO:teuthology.orchestra.run.trial158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:01.702 INFO:teuthology.orchestra.run.trial158.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:01.702 INFO:teuthology.orchestra.run.trial158.stdout:Access: 2026-02-25 08:15:59.409100154 +0000 2026-02-25T08:40:01.702 INFO:teuthology.orchestra.run.trial158.stdout:Modify: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.702 INFO:teuthology.orchestra.run.trial158.stdout:Change: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.703 INFO:teuthology.orchestra.run.trial158.stdout: Birth: 2026-02-25 08:07:05.428636862 +0000 2026-02-25T08:40:01.703 DEBUG:teuthology.orchestra.run.trial158:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T08:40:01.764 INFO:teuthology.orchestra.run.trial158.stderr:1+0 records in 2026-02-25T08:40:01.764 INFO:teuthology.orchestra.run.trial158.stderr:1+0 records out 2026-02-25T08:40:01.764 INFO:teuthology.orchestra.run.trial158.stderr:512 bytes copied, 0.00016508 s, 3.1 MB/s 2026-02-25T08:40:01.766 DEBUG:teuthology.orchestra.run.trial158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T08:40:01.823 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:01.823 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial006.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial158.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-25T08:40:01.823 INFO:tasks.ceph:Generating config... 2026-02-25T08:40:01.825 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-25T08:40:01.825 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-25T08:40:01.825 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-25T08:40:01.825 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-25T08:40:01.826 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] osd delete sleep = 1 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] osd max backfills = 3 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-25T08:40:01.827 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2026-02-25T08:40:01.828 INFO:tasks.ceph:[osd] osd target transaction size = 5 2026-02-25T08:40:01.828 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-25T08:40:01.828 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-25T08:40:01.828 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-25T08:40:01.828 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-25T08:40:01.828 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-25T08:40:01.828 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-25T08:40:01.828 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-25T08:40:01.828 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-25T08:40:01.828 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-25T08:40:01.828 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-25T08:40:01.828 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2026-02-25T08:40:01.828 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2026-02-25T08:40:01.829 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-25T08:40:01.829 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-25T08:40:01.829 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-25T08:40:01.829 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-25T08:40:01.829 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-25T08:40:01.829 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-25T08:40:01.829 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-25T08:40:01.829 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-25T08:40:01.829 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-25T08:40:01.829 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-25T08:40:01.829 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-25T08:40:01.830 INFO:tasks.ceph:Setting up mon.a... 2026-02-25T08:40:01.830 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-25T08:40:01.872 INFO:teuthology.orchestra.run.trial006.stdout:creating /etc/ceph/ceph.keyring 2026-02-25T08:40:01.874 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-25T08:40:01.957 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T08:40:02.021 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.6')] 2026-02-25T08:40:02.022 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.6', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd target transaction size': 5, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-25T08:40:02.023 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:40:02.023 DEBUG:teuthology.orchestra.run.trial006:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-25T08:40:02.077 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.6 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:02.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:02.153+0000 7f1d34030f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:02.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:02.153+0000 7f1d34030f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:02.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:02.153+0000 7f1d34030f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:02.157 INFO:teuthology.orchestra.run.trial006.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:02.157 INFO:teuthology.orchestra.run.trial006.stdout:monmaptool: generated fsid b1b76914-ff72-4771-8729-7fc8b1bbfc83 2026-02-25T08:40:02.157 INFO:teuthology.orchestra.run.trial006.stdout:setting min_mon_release = tentacle 2026-02-25T08:40:02.157 INFO:teuthology.orchestra.run.trial006.stdout:epoch 0 2026-02-25T08:40:02.157 INFO:teuthology.orchestra.run.trial006.stdout:fsid b1b76914-ff72-4771-8729-7fc8b1bbfc83 2026-02-25T08:40:02.157 INFO:teuthology.orchestra.run.trial006.stdout:last_changed 2026-02-25T08:40:02.155548+0000 2026-02-25T08:40:02.158 INFO:teuthology.orchestra.run.trial006.stdout:created 2026-02-25T08:40:02.155548+0000 2026-02-25T08:40:02.158 INFO:teuthology.orchestra.run.trial006.stdout:min_mon_release 20 (tentacle) 2026-02-25T08:40:02.158 INFO:teuthology.orchestra.run.trial006.stdout:election_strategy: 1 2026-02-25T08:40:02.158 INFO:teuthology.orchestra.run.trial006.stdout:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-25T08:40:02.158 INFO:teuthology.orchestra.run.trial006.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-25T08:40:02.159 DEBUG:teuthology.orchestra.run.trial006:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-25T08:40:02.213 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b1b76914-ff72-4771-8729-7fc8b1bbfc83... 2026-02-25T08:40:02.214 DEBUG:teuthology.orchestra.run.trial006:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-25T08:40:02.255 DEBUG:teuthology.orchestra.run.trial158:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-25T08:40:02.294 INFO:teuthology.orchestra.run.trial006.stdout:[global] 2026-02-25T08:40:02.294 INFO:teuthology.orchestra.run.trial006.stdout: chdir = "" 2026-02-25T08:40:02.294 INFO:teuthology.orchestra.run.trial006.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-25T08:40:02.294 INFO:teuthology.orchestra.run.trial006.stdout: auth supported = cephx 2026-02-25T08:40:02.294 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.294 INFO:teuthology.orchestra.run.trial006.stdout: filestore xattr use omap = true 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: mon clock drift allowed = 1.000 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: osd crush chooseleaf type = 0 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: auth debug = true 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: ms die on old message = true 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: ms die on bug = true 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: mon pg warn max object skew = 0 2026-02-25T08:40:02.295 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: # disable pg_autoscaler by default for new pools 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: osd pool default size = 2 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: mon osd allow primary affinity = true 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: mon osd allow pg remap = true 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: mon warn on legacy crush tunables = false 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: mon warn on crush straw calc version zero = false 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: mon warn on no sortbitwise = false 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: mon warn on osd down out interval zero = false 2026-02-25T08:40:02.296 INFO:teuthology.orchestra.run.trial006.stdout: mon warn on too few osds = false 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: mon_allow_pool_size_one = true 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: osd default data pool replay window = 5 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: mon allow pool delete = true 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: mon cluster log file level = debug 2026-02-25T08:40:02.297 INFO:teuthology.orchestra.run.trial006.stdout: debug asserts on shutdown = true 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: mon health detail to clog = false 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: mon host = 10.20.193.6 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: ms client mode = crc 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: ms cluster mode = crc 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: ms mon client mode = crc 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: ms mon cluster mode = crc 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: ms mon service mode = crc 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: ms service mode = crc 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: osd_pool_default_min_size = 2 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: osd_pool_default_size = 3 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: fsid = b1b76914-ff72-4771-8729-7fc8b1bbfc83 2026-02-25T08:40:02.298 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout:[osd] 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: osd journal size = 100 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: osd scrub load threshold = 5.0 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: osd scrub max interval = 120 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: osd mclock profile = high_recovery_ops 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: osd mclock skip benchmark = true 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: osd recover clone overlap = true 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: osd recovery max chunk = 1048576 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: osd debug shutdown = true 2026-02-25T08:40:02.299 INFO:teuthology.orchestra.run.trial006.stdout: osd debug op order = true 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: osd debug verify stray on activate = true 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: osd debug trim objects = true 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: osd open classes on start = true 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: osd debug pg log writeout = true 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: osd deep scrub update digest min age = 30 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: osd map max advance = 10 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: journal zero on create = true 2026-02-25T08:40:02.300 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: filestore ondisk finisher threads = 3 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: filestore apply finisher threads = 3 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: bdev debug aio = true 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: osd debug misdirected ops = true 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: debug monc = 20 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: crimson cpu num = 4 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: crimson osd obc lru size = 10 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: debug ms = 20 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: debug osd = 20 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore = 20 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore backref = 20 2026-02-25T08:40:02.301 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore cache = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore cleaner = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore device = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore epm = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore fixedkv tree = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore journal = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore lba = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore odata = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore omap = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore onode = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore t = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: debug seastore tm = 20 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: osd blocked scrub grace period = 3600 2026-02-25T08:40:02.302 INFO:teuthology.orchestra.run.trial006.stdout: osd debug reject backfill probability = 0.3 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: osd delete sleep = 1 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: osd max backfills = 3 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: osd max markdown count = 1000 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: osd objectstore = seastore 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: osd scrub min interval = 60 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: osd shutdown pgref assert = True 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: osd snap trim sleep = 2 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: osd target transaction size = 5 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: seastore cachepin size pershard = 64M 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: seastore cachepin type = 2Q 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: seastore max concurrent transactions = 8 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.303 INFO:teuthology.orchestra.run.trial006.stdout:[mgr] 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: debug ms = 1 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: debug mgr = 20 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: debug mon = 20 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: debug auth = 20 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: mon reweight min pgs per osd = 4 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: mon reweight min bytes per osd = 10 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: mgr/telemetry/nag = false 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout:[mon] 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: debug ms = 1 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: debug mon = 20 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: debug paxos = 20 2026-02-25T08:40:02.304 INFO:teuthology.orchestra.run.trial006.stdout: debug auth = 20 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: mon data avail warn = 5 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: mon mgr mkfs grace = 240 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: mon reweight min pgs per osd = 4 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: mon osd reporter subtree level = osd 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: mon osd prime pg temp = true 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: mon reweight min bytes per osd = 10 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: auth mon ticket ttl = 660 # 11m 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: auth service ticket ttl = 240 # 4m 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.305 INFO:teuthology.orchestra.run.trial006.stdout: # don't complain about insecure global_id in the test suite 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: # 1m isn't quite enough 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: mon_down_mkfs_grace = 2m 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: mon_warn_on_filestore_osds = false 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: mon min osdmap epochs = 50 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: mon osdmap full prune interval = 2 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: mon osdmap full prune min = 15 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: mon osdmap full prune txsize = 2 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: osd pool default crimson = True 2026-02-25T08:40:02.306 INFO:teuthology.orchestra.run.trial006.stdout: paxos service trim min = 10 2026-02-25T08:40:02.307 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:02.307 INFO:teuthology.orchestra.run.trial006.stdout:[client] 2026-02-25T08:40:02.307 INFO:teuthology.orchestra.run.trial006.stdout: rgw cache enabled = true 2026-02-25T08:40:02.307 INFO:teuthology.orchestra.run.trial006.stdout: rgw enable ops log = true 2026-02-25T08:40:02.307 INFO:teuthology.orchestra.run.trial006.stdout: rgw enable usage log = true 2026-02-25T08:40:02.307 INFO:teuthology.orchestra.run.trial006.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-25T08:40:02.307 INFO:teuthology.orchestra.run.trial006.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-25T08:40:02.307 INFO:teuthology.orchestra.run.trial006.stdout:[mon.a] 2026-02-25T08:40:02.309 INFO:teuthology.orchestra.run.trial158.stdout:[global] 2026-02-25T08:40:02.309 INFO:teuthology.orchestra.run.trial158.stdout: chdir = "" 2026-02-25T08:40:02.309 INFO:teuthology.orchestra.run.trial158.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-25T08:40:02.309 INFO:teuthology.orchestra.run.trial158.stdout: auth supported = cephx 2026-02-25T08:40:02.309 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.309 INFO:teuthology.orchestra.run.trial158.stdout: filestore xattr use omap = true 2026-02-25T08:40:02.309 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.310 INFO:teuthology.orchestra.run.trial158.stdout: mon clock drift allowed = 1.000 2026-02-25T08:40:02.310 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.310 INFO:teuthology.orchestra.run.trial158.stdout: osd crush chooseleaf type = 0 2026-02-25T08:40:02.310 INFO:teuthology.orchestra.run.trial158.stdout: auth debug = true 2026-02-25T08:40:02.310 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.310 INFO:teuthology.orchestra.run.trial158.stdout: ms die on old message = true 2026-02-25T08:40:02.310 INFO:teuthology.orchestra.run.trial158.stdout: ms die on bug = true 2026-02-25T08:40:02.310 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.310 INFO:teuthology.orchestra.run.trial158.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-25T08:40:02.310 INFO:teuthology.orchestra.run.trial158.stdout: mon pg warn max object skew = 0 2026-02-25T08:40:02.310 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: # disable pg_autoscaler by default for new pools 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: osd pool default size = 2 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: mon osd allow primary affinity = true 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: mon osd allow pg remap = true 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: mon warn on legacy crush tunables = false 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: mon warn on crush straw calc version zero = false 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: mon warn on no sortbitwise = false 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: mon warn on osd down out interval zero = false 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: mon warn on too few osds = false 2026-02-25T08:40:02.311 INFO:teuthology.orchestra.run.trial158.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: mon_allow_pool_size_one = true 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: osd default data pool replay window = 5 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: mon allow pool delete = true 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: mon cluster log file level = debug 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: debug asserts on shutdown = true 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: mon health detail to clog = false 2026-02-25T08:40:02.312 INFO:teuthology.orchestra.run.trial158.stdout: mon host = 10.20.193.6 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout: ms client mode = crc 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout: ms cluster mode = crc 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout: ms mon client mode = crc 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout: ms mon cluster mode = crc 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout: ms mon service mode = crc 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout: ms service mode = crc 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout: osd_pool_default_min_size = 2 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout: osd_pool_default_size = 3 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout: fsid = b1b76914-ff72-4771-8729-7fc8b1bbfc83 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.313 INFO:teuthology.orchestra.run.trial158.stdout:[osd] 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: osd journal size = 100 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: osd scrub load threshold = 5.0 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: osd scrub max interval = 120 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: osd mclock profile = high_recovery_ops 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: osd mclock skip benchmark = true 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: osd recover clone overlap = true 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: osd recovery max chunk = 1048576 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: osd debug shutdown = true 2026-02-25T08:40:02.314 INFO:teuthology.orchestra.run.trial158.stdout: osd debug op order = true 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: osd debug verify stray on activate = true 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: osd debug trim objects = true 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: osd open classes on start = true 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: osd debug pg log writeout = true 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: osd deep scrub update digest min age = 30 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: osd map max advance = 10 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: journal zero on create = true 2026-02-25T08:40:02.315 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: filestore ondisk finisher threads = 3 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: filestore apply finisher threads = 3 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: bdev debug aio = true 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: osd debug misdirected ops = true 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: debug monc = 20 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: crimson cpu num = 4 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: crimson osd obc lru size = 10 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: debug ms = 20 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: debug osd = 20 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore = 20 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore backref = 20 2026-02-25T08:40:02.316 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore cache = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore cleaner = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore device = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore epm = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore fixedkv tree = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore journal = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore lba = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore odata = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore omap = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore onode = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore t = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: debug seastore tm = 20 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: osd blocked scrub grace period = 3600 2026-02-25T08:40:02.317 INFO:teuthology.orchestra.run.trial158.stdout: osd debug reject backfill probability = 0.3 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: osd delete sleep = 1 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: osd max backfills = 3 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: osd max markdown count = 1000 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: osd objectstore = seastore 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: osd scrub min interval = 60 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: osd shutdown pgref assert = True 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: osd snap trim sleep = 2 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: osd target transaction size = 5 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: seastore cachepin size pershard = 64M 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: seastore cachepin type = 2Q 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: seastore max concurrent transactions = 8 2026-02-25T08:40:02.318 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout:[mgr] 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout: debug ms = 1 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout: debug mgr = 20 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout: debug mon = 20 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout: debug auth = 20 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout: mon reweight min pgs per osd = 4 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout: mon reweight min bytes per osd = 10 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout: mgr/telemetry/nag = false 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout:[mon] 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout: debug ms = 1 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout: debug mon = 20 2026-02-25T08:40:02.319 INFO:teuthology.orchestra.run.trial158.stdout: debug paxos = 20 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: debug auth = 20 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: mon data avail warn = 5 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: mon mgr mkfs grace = 240 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: mon reweight min pgs per osd = 4 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: mon osd reporter subtree level = osd 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: mon osd prime pg temp = true 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: mon reweight min bytes per osd = 10 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: auth mon ticket ttl = 660 # 11m 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: auth service ticket ttl = 240 # 4m 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.320 INFO:teuthology.orchestra.run.trial158.stdout: # don't complain about insecure global_id in the test suite 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: # 1m isn't quite enough 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: mon_down_mkfs_grace = 2m 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: mon_warn_on_filestore_osds = false 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: mon min osdmap epochs = 50 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: mon osdmap full prune interval = 2 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: mon osdmap full prune min = 15 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: mon osdmap full prune txsize = 2 2026-02-25T08:40:02.321 INFO:teuthology.orchestra.run.trial158.stdout: osd pool default crimson = True 2026-02-25T08:40:02.322 INFO:teuthology.orchestra.run.trial158.stdout: paxos service trim min = 10 2026-02-25T08:40:02.322 INFO:teuthology.orchestra.run.trial158.stdout: 2026-02-25T08:40:02.322 INFO:teuthology.orchestra.run.trial158.stdout:[client] 2026-02-25T08:40:02.322 INFO:teuthology.orchestra.run.trial158.stdout: rgw cache enabled = true 2026-02-25T08:40:02.322 INFO:teuthology.orchestra.run.trial158.stdout: rgw enable ops log = true 2026-02-25T08:40:02.322 INFO:teuthology.orchestra.run.trial158.stdout: rgw enable usage log = true 2026-02-25T08:40:02.322 INFO:teuthology.orchestra.run.trial158.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-25T08:40:02.322 INFO:teuthology.orchestra.run.trial158.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-25T08:40:02.322 INFO:teuthology.orchestra.run.trial158.stdout:[mon.a] 2026-02-25T08:40:02.322 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-25T08:40:02.323 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-25T08:40:02.398 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-25T08:40:02.399 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:40:02.399 DEBUG:teuthology.orchestra.run.trial006:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-25T08:40:02.415 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:40:02.415 DEBUG:teuthology.orchestra.run.trial006:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-25T08:40:02.471 INFO:tasks.ceph:Sending monmap to node ubuntu@trial006.front.sepia.ceph.com 2026-02-25T08:40:02.471 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:40:02.472 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-25T08:40:02.472 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T08:40:02.545 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:40:02.545 DEBUG:teuthology.orchestra.run.trial006:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:02.601 INFO:tasks.ceph:Sending monmap to node ubuntu@trial158.front.sepia.ceph.com 2026-02-25T08:40:02.601 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:40:02.602 DEBUG:teuthology.orchestra.run.trial158:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-25T08:40:02.602 DEBUG:teuthology.orchestra.run.trial158:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T08:40:02.635 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:40:02.635 DEBUG:teuthology.orchestra.run.trial158:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:02.691 INFO:tasks.ceph:Setting up mon nodes... 2026-02-25T08:40:02.691 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-25T08:40:02.691 DEBUG:teuthology.orchestra.run.trial158:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-25T08:40:02.780 INFO:teuthology.orchestra.run.trial158.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-25T08:40:02.783 INFO:tasks.ceph:Setting up mds nodes... 2026-02-25T08:40:02.783 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-25T08:40:02.783 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-25T08:40:02.826 INFO:teuthology.orchestra.run.trial006.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-25T08:40:02.837 DEBUG:teuthology.orchestra.run.trial158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2026-02-25T08:40:02.878 INFO:teuthology.orchestra.run.trial158.stdout:creating /etc/ceph/ceph.client.1.keyring 2026-02-25T08:40:02.889 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-25T08:40:02.890 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial006.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial158.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-25T08:40:02.890 DEBUG:teuthology.orchestra.run.trial006:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:02.915 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:02.915 INFO:tasks.ceph:role: osd.0 2026-02-25T08:40:02.916 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial006.front.sepia.ceph.com 2026-02-25T08:40:02.916 DEBUG:teuthology.orchestra.run.trial006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-25T08:40:02.980 INFO:teuthology.orchestra.run.trial006.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T08:40:02.980 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T08:40:02.980 INFO:teuthology.orchestra.run.trial006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T08:40:02.980 INFO:teuthology.orchestra.run.trial006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T08:40:02.981 INFO:teuthology.orchestra.run.trial006.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T08:40:02.981 INFO:teuthology.orchestra.run.trial006.stdout: = sunit=0 swidth=0 blks 2026-02-25T08:40:02.981 INFO:teuthology.orchestra.run.trial006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T08:40:02.981 INFO:teuthology.orchestra.run.trial006.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T08:40:02.981 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T08:40:02.981 INFO:teuthology.orchestra.run.trial006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T08:40:03.562 INFO:teuthology.orchestra.run.trial006.stdout:Discarding blocks...Done. 2026-02-25T08:40:03.564 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial006.front.sepia.ceph.com -o noatime 2026-02-25T08:40:03.565 DEBUG:teuthology.orchestra.run.trial006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:03.633 DEBUG:teuthology.orchestra.run.trial006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:03.701 DEBUG:teuthology.orchestra.run.trial006:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-25T08:40:03.765 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:03.765 INFO:tasks.ceph:role: osd.1 2026-02-25T08:40:03.765 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial006.front.sepia.ceph.com 2026-02-25T08:40:03.765 DEBUG:teuthology.orchestra.run.trial006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-25T08:40:03.830 INFO:teuthology.orchestra.run.trial006.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T08:40:03.830 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T08:40:03.830 INFO:teuthology.orchestra.run.trial006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T08:40:03.830 INFO:teuthology.orchestra.run.trial006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T08:40:03.830 INFO:teuthology.orchestra.run.trial006.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T08:40:03.830 INFO:teuthology.orchestra.run.trial006.stdout: = sunit=0 swidth=0 blks 2026-02-25T08:40:03.830 INFO:teuthology.orchestra.run.trial006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T08:40:03.830 INFO:teuthology.orchestra.run.trial006.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T08:40:03.831 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T08:40:03.831 INFO:teuthology.orchestra.run.trial006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T08:40:04.389 INFO:teuthology.orchestra.run.trial006.stdout:Discarding blocks...Done. 2026-02-25T08:40:04.391 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial006.front.sepia.ceph.com -o noatime 2026-02-25T08:40:04.391 DEBUG:teuthology.orchestra.run.trial006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-25T08:40:04.459 DEBUG:teuthology.orchestra.run.trial006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-25T08:40:04.525 DEBUG:teuthology.orchestra.run.trial006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:04.675 INFO:teuthology.orchestra.run.trial006.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T08:40:04.728 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:04,728 seastar - Reactor backend: linux-aio 2026-02-25T08:40:04.745 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:04,745 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:04.746 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:04,745 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:04.748 INFO:teuthology.orchestra.run.trial006.stderr:==40035==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:04.804 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:04,803 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T08:40:04.804 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:04,803 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T08:40:04.804 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:04,803 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T08:40:04.804 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:04,803 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T08:40:04.804 INFO:teuthology.orchestra.run.trial006.stderr:DEBUG 2026-02-25 08:40:04,803 [shard 0:main] seastar - reactor::drain 2026-02-25T08:40:06.848 INFO:teuthology.orchestra.run.trial006.stderr:----------------------------------------------------- 2026-02-25T08:40:06.848 INFO:teuthology.orchestra.run.trial006.stderr:Suppressions used: 2026-02-25T08:40:06.848 INFO:teuthology.orchestra.run.trial006.stderr: count bytes template 2026-02-25T08:40:06.848 INFO:teuthology.orchestra.run.trial006.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:06.848 INFO:teuthology.orchestra.run.trial006.stderr:----------------------------------------------------- 2026-02-25T08:40:06.848 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-25T08:40:06.866 INFO:teuthology.orchestra.run.trial006.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T08:40:06.867 INFO:teuthology.orchestra.run.trial006.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T08:40:06.914 INFO:teuthology.orchestra.run.trial006.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T08:40:06.914 INFO:teuthology.orchestra.run.trial006.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T08:40:06.921 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:06,921 seastar - Reactor backend: linux-aio 2026-02-25T08:40:06.933 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:06,933 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:06.933 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:06,933 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:06.938 INFO:teuthology.orchestra.run.trial006.stderr:==40033==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:06.938 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:06,937 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T08:40:06.998 INFO:teuthology.orchestra.run.trial006.stderr:DEBUG 2026-02-25 08:40:06,997 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T08:40:07.079 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x11380c8 0xf4792d 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e22e1c 0x9e2ae9f 0x9e2b21b 0x9f13d68 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:07.079 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb04e6 /lib64/libasan.so.8+0xb1437 0xb7ac87 0x10a27ac 0xf47938 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9e24090 0x9e2428f 0x9f095b1 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:07.079 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x10a26bf 0xf47938 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e220d2 0x9e22694 0x9e2ab3b 0x9e2b21b 0x9f13d68 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:07.111 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x9ef525d 0x9ef558c 0x9ef57f6 0x9ef6a80 0x9ef6ea0 0x9f1372b 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba26b2 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7045030 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:07.181 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a6955f 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:07.187 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a6905f 0x7d56d00 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:07.219 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15192 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:07.219 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:07.342 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0xcc2652 0x9f1a3b4 0x78b69db 0x7ba11e3 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:07.342 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x11380c8 0xf4792d 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e220d2 0x9e22b81 0x9e2307f 0x9e236af 0x9e23819 0x9f03b89 0x9f1698c 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:07.342 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x742b42e 0x731ea29 0x78b693f 0x7ba17e8 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:07.424 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a68a34 0x7d56cf3 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:07.440 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:07.440 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a1526e 0x2a69674 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:07.442 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65d9 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:09.631 INFO:teuthology.orchestra.run.trial006.stderr:----------------------------------------------------- 2026-02-25T08:40:09.631 INFO:teuthology.orchestra.run.trial006.stderr:Suppressions used: 2026-02-25T08:40:09.631 INFO:teuthology.orchestra.run.trial006.stderr: count bytes template 2026-02-25T08:40:09.631 INFO:teuthology.orchestra.run.trial006.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:09.631 INFO:teuthology.orchestra.run.trial006.stderr:----------------------------------------------------- 2026-02-25T08:40:09.631 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-25T08:40:09.668 DEBUG:teuthology.orchestra.run.trial006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:09.694 DEBUG:teuthology.orchestra.run.trial006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:09.847 INFO:teuthology.orchestra.run.trial006.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T08:40:09.900 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:09,899 seastar - Reactor backend: linux-aio 2026-02-25T08:40:09.913 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:09,913 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:09.913 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:09,913 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:09.915 INFO:teuthology.orchestra.run.trial006.stderr:==40101==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:09.972 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:09,971 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T08:40:09.972 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:09,971 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T08:40:09.972 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:09,971 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T08:40:09.972 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:09,971 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T08:40:09.972 INFO:teuthology.orchestra.run.trial006.stderr:DEBUG 2026-02-25 08:40:09,971 [shard 0:main] seastar - reactor::drain 2026-02-25T08:40:12.028 INFO:teuthology.orchestra.run.trial006.stderr:----------------------------------------------------- 2026-02-25T08:40:12.028 INFO:teuthology.orchestra.run.trial006.stderr:Suppressions used: 2026-02-25T08:40:12.028 INFO:teuthology.orchestra.run.trial006.stderr: count bytes template 2026-02-25T08:40:12.028 INFO:teuthology.orchestra.run.trial006.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:12.028 INFO:teuthology.orchestra.run.trial006.stderr:----------------------------------------------------- 2026-02-25T08:40:12.028 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-25T08:40:12.047 INFO:teuthology.orchestra.run.trial006.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T08:40:12.047 INFO:teuthology.orchestra.run.trial006.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T08:40:12.088 INFO:teuthology.orchestra.run.trial006.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T08:40:12.089 INFO:teuthology.orchestra.run.trial006.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T08:40:12.096 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:12,096 seastar - Reactor backend: linux-aio 2026-02-25T08:40:12.112 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:12,111 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:12.112 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:12,111 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:12.115 INFO:teuthology.orchestra.run.trial006.stderr:==40099==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:12.116 INFO:teuthology.orchestra.run.trial006.stderr:INFO 2026-02-25 08:40:12,115 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T08:40:12.174 INFO:teuthology.orchestra.run.trial006.stderr:DEBUG 2026-02-25 08:40:12,174 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T08:40:12.255 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xedbe8 0xe40f5d 0x9d754d7 0x9eec474 0x9eec56d 0x7ba1220 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:12.256 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1583b 0x10a2823 0xf47938 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9e24090 0x9e2428f 0x9f095b1 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:12.256 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x318123b 0x9e21b32 0x9e21e1c 0x9e220d2 0x9e22b81 0x9e2307f 0x9e236af 0x9e23819 0x9f03b89 0x9f1698c 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:12.288 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x8c9ac5d 0x8f526d3 0x72bece8 0x711843b 0x6f40d7e 0x723b16f 0x7162a9c 0x7045030 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:12.355 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a684bb 0x7d56cde 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:12.396 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c6 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:12.396 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a695a0 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:12.396 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc08 /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:12.520 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0xf4797a 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e220d2 0x9e22694 0x9e2ab3b 0x9e2b21b 0x9f13d68 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:12.521 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cbef /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xfd19a /lib64/libasan.so.8+0xedcb5 0xe2f10b 0xe2f42e 0x9ef6eb8 0x9f1372b 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:12.521 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119155 /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xec776 0xcc26c1 0x9ef69f3 0x9ef6ea0 0x9f1372b 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:12.603 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a1526e 0x2a69674 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:12.619 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a68fa1 0x7d56d00 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:12.619 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:12.619 INFO:teuthology.orchestra.run.trial006.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:14.772 INFO:teuthology.orchestra.run.trial006.stderr:----------------------------------------------------- 2026-02-25T08:40:14.772 INFO:teuthology.orchestra.run.trial006.stderr:Suppressions used: 2026-02-25T08:40:14.772 INFO:teuthology.orchestra.run.trial006.stderr: count bytes template 2026-02-25T08:40:14.773 INFO:teuthology.orchestra.run.trial006.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:14.773 INFO:teuthology.orchestra.run.trial006.stderr:----------------------------------------------------- 2026-02-25T08:40:14.773 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-25T08:40:14.808 DEBUG:teuthology.orchestra.run.trial006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-25T08:40:14.833 DEBUG:teuthology.orchestra.run.trial158:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-25T08:40:14.858 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:14.858 INFO:tasks.ceph:role: osd.2 2026-02-25T08:40:14.859 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial158.front.sepia.ceph.com 2026-02-25T08:40:14.859 DEBUG:teuthology.orchestra.run.trial158:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-25T08:40:14.922 INFO:teuthology.orchestra.run.trial158.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T08:40:14.923 INFO:teuthology.orchestra.run.trial158.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T08:40:14.923 INFO:teuthology.orchestra.run.trial158.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T08:40:14.923 INFO:teuthology.orchestra.run.trial158.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T08:40:14.923 INFO:teuthology.orchestra.run.trial158.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T08:40:14.923 INFO:teuthology.orchestra.run.trial158.stdout: = sunit=0 swidth=0 blks 2026-02-25T08:40:14.923 INFO:teuthology.orchestra.run.trial158.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T08:40:14.923 INFO:teuthology.orchestra.run.trial158.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T08:40:14.923 INFO:teuthology.orchestra.run.trial158.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T08:40:14.923 INFO:teuthology.orchestra.run.trial158.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T08:40:15.496 INFO:teuthology.orchestra.run.trial158.stdout:Discarding blocks...Done. 2026-02-25T08:40:15.498 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial158.front.sepia.ceph.com -o noatime 2026-02-25T08:40:15.499 DEBUG:teuthology.orchestra.run.trial158:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2026-02-25T08:40:15.568 DEBUG:teuthology.orchestra.run.trial158:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-25T08:40:15.635 DEBUG:teuthology.orchestra.run.trial158:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-25T08:40:15.698 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:15.699 INFO:tasks.ceph:role: osd.3 2026-02-25T08:40:15.699 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial158.front.sepia.ceph.com 2026-02-25T08:40:15.699 DEBUG:teuthology.orchestra.run.trial158:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-25T08:40:15.764 INFO:teuthology.orchestra.run.trial158.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T08:40:15.764 INFO:teuthology.orchestra.run.trial158.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T08:40:15.764 INFO:teuthology.orchestra.run.trial158.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T08:40:15.764 INFO:teuthology.orchestra.run.trial158.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T08:40:15.765 INFO:teuthology.orchestra.run.trial158.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T08:40:15.765 INFO:teuthology.orchestra.run.trial158.stdout: = sunit=0 swidth=0 blks 2026-02-25T08:40:15.765 INFO:teuthology.orchestra.run.trial158.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T08:40:15.765 INFO:teuthology.orchestra.run.trial158.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T08:40:15.765 INFO:teuthology.orchestra.run.trial158.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T08:40:15.765 INFO:teuthology.orchestra.run.trial158.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T08:40:16.324 INFO:teuthology.orchestra.run.trial158.stdout:Discarding blocks...Done. 2026-02-25T08:40:16.326 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial158.front.sepia.ceph.com -o noatime 2026-02-25T08:40:16.326 DEBUG:teuthology.orchestra.run.trial158:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2026-02-25T08:40:16.394 DEBUG:teuthology.orchestra.run.trial158:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-25T08:40:16.460 DEBUG:teuthology.orchestra.run.trial158:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:16.612 INFO:teuthology.orchestra.run.trial158.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T08:40:16.666 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:16,664 seastar - Reactor backend: linux-aio 2026-02-25T08:40:16.677 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:16,674 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:16.677 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:16,674 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:16.679 INFO:teuthology.orchestra.run.trial158.stderr:==39515==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:16.736 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:16,733 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T08:40:16.736 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:16,733 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T08:40:16.736 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:16,733 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T08:40:16.736 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:16,733 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T08:40:16.736 INFO:teuthology.orchestra.run.trial158.stderr:DEBUG 2026-02-25 08:40:16,734 [shard 0:main] seastar - reactor::drain 2026-02-25T08:40:18.799 INFO:teuthology.orchestra.run.trial158.stderr:----------------------------------------------------- 2026-02-25T08:40:18.799 INFO:teuthology.orchestra.run.trial158.stderr:Suppressions used: 2026-02-25T08:40:18.800 INFO:teuthology.orchestra.run.trial158.stderr: count bytes template 2026-02-25T08:40:18.800 INFO:teuthology.orchestra.run.trial158.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:18.800 INFO:teuthology.orchestra.run.trial158.stderr:----------------------------------------------------- 2026-02-25T08:40:18.800 INFO:teuthology.orchestra.run.trial158.stderr: 2026-02-25T08:40:18.819 INFO:teuthology.orchestra.run.trial158.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T08:40:18.819 INFO:teuthology.orchestra.run.trial158.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T08:40:18.860 INFO:teuthology.orchestra.run.trial158.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T08:40:18.860 INFO:teuthology.orchestra.run.trial158.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T08:40:18.867 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:18,865 seastar - Reactor backend: linux-aio 2026-02-25T08:40:18.876 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:18,873 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:18.876 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:18,873 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:18.880 INFO:teuthology.orchestra.run.trial158.stderr:==39513==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:18.880 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:18,877 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T08:40:18.941 INFO:teuthology.orchestra.run.trial158.stderr:DEBUG 2026-02-25 08:40:18,938 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T08:40:19.025 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x35fbd18 0x35fbfe6 0x361a32a 0x361a79f 0x361a859 0x9ef6a64 0x9ef6ea0 0x9f1372b 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.025 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb057a /lib64/libasan.so.8+0xb1437 0xb7ac87 0x10a27ac 0xf47938 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9f09a9d 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.025 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x9ef403b 0x9ef5765 0x9ef6a80 0x9ef6ea0 0x9f1372b 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.057 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191ccd /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x782d17a 0x9558c65 0x959e7f8 0x962a7bc 0x9c37711 0x9646bcf 0x9646d30 0x7a5996a 0x716287f 0x7045030 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:19.057 INFO:teuthology.orchestra.run.trial158.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff9b891052 0xffffffff9c550451 0xffffffff9c600ba2 2026-02-25T08:40:19.100 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb0470 /lib64/libasan.so.8+0xb1437 0xa7a7c76 0x9f193cc 0x9f195b3 0x9f1977c 0x9f1980f 0x9f1a654 0x78b69db 0x7b90cb1 0x7ba55d5 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7045030 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.100 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xeacf /lib64/libasan.so.8+0xec776 0xcc26c1 0x6f43408 0x9ef901e 0x9ef9236 0x9ef9508 0x7f64096 0x7fb93c0 0x9121733 0x91240a1 0x7857565 0x7a67c92 0x704540f 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.124 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a69662 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:19.165 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc25 /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.165 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a6905f 0x7d56d00 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.168 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15277 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.297 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x7c9e538 0x7ba148d 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.297 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0xe29e0e 0x1138071 0xf4792d 0x9f044e9 0x9f04ad6 0x9f04e59 0x9f14fb0 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.298 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9cf61eb 0x9d73986 0x9d73a9c 0x9d73b78 0x9d73df4 0x9d73ead 0x9d7439b 0x9d7479f 0x9d74a5c 0x9f165e1 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.298 INFO:teuthology.orchestra.run.trial158.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff9ba1ff44 0xffffffff9ba203b0 0xffffffff9b89525e 0xffffffff9bbe7269 0xffffffff9bbca7d5 0xffffffff9bbd4bab 0xffffffff9bbd500b 0xffffffff9b8910d2 0xffffffff9c550451 0xffffffff9c600ba2 2026-02-25T08:40:19.380 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a1526e 0x2a69674 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:19.395 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a695b6 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.396 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a686a4 0x7d56cde 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:19.399 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ed3 /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:21.565 INFO:teuthology.orchestra.run.trial158.stderr:----------------------------------------------------- 2026-02-25T08:40:21.565 INFO:teuthology.orchestra.run.trial158.stderr:Suppressions used: 2026-02-25T08:40:21.565 INFO:teuthology.orchestra.run.trial158.stderr: count bytes template 2026-02-25T08:40:21.565 INFO:teuthology.orchestra.run.trial158.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:21.566 INFO:teuthology.orchestra.run.trial158.stderr:----------------------------------------------------- 2026-02-25T08:40:21.566 INFO:teuthology.orchestra.run.trial158.stderr: 2026-02-25T08:40:21.599 DEBUG:teuthology.orchestra.run.trial158:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-25T08:40:21.624 DEBUG:teuthology.orchestra.run.trial158:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:21.775 INFO:teuthology.orchestra.run.trial158.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T08:40:21.835 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:21,832 seastar - Reactor backend: linux-aio 2026-02-25T08:40:21.852 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:21,850 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:21.852 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:21,850 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:21.854 INFO:teuthology.orchestra.run.trial158.stderr:==39582==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:21.911 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:21,908 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T08:40:21.911 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:21,908 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T08:40:21.911 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:21,908 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T08:40:21.911 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:21,908 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T08:40:21.911 INFO:teuthology.orchestra.run.trial158.stderr:DEBUG 2026-02-25 08:40:21,908 [shard 0:main] seastar - reactor::drain 2026-02-25T08:40:23.967 INFO:teuthology.orchestra.run.trial158.stderr:----------------------------------------------------- 2026-02-25T08:40:23.967 INFO:teuthology.orchestra.run.trial158.stderr:Suppressions used: 2026-02-25T08:40:23.967 INFO:teuthology.orchestra.run.trial158.stderr: count bytes template 2026-02-25T08:40:23.968 INFO:teuthology.orchestra.run.trial158.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:23.968 INFO:teuthology.orchestra.run.trial158.stderr:----------------------------------------------------- 2026-02-25T08:40:23.968 INFO:teuthology.orchestra.run.trial158.stderr: 2026-02-25T08:40:23.987 INFO:teuthology.orchestra.run.trial158.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T08:40:23.987 INFO:teuthology.orchestra.run.trial158.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T08:40:24.028 INFO:teuthology.orchestra.run.trial158.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T08:40:24.029 INFO:teuthology.orchestra.run.trial158.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T08:40:24.035 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:24,033 seastar - Reactor backend: linux-aio 2026-02-25T08:40:24.046 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:24,044 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:24.046 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:24,044 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:24.050 INFO:teuthology.orchestra.run.trial158.stderr:==39580==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:24.050 INFO:teuthology.orchestra.run.trial158.stderr:INFO 2026-02-25 08:40:24,048 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T08:40:24.108 INFO:teuthology.orchestra.run.trial158.stderr:DEBUG 2026-02-25 08:40:24,106 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T08:40:24.189 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x157c7 0x742b1ed 0x742b303 0x725c067 0x711b842 0x7ba1535 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.190 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x16c4f 0x9f0789c 0x9f0916c 0x9f097c6 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.190 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x11380c8 0xf4792d 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9e24090 0x9e2428f 0x9f095b1 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.222 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x157c7 0xca7484 0xcb9527 0x9e6a9dc 0x9f13f8f 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba64cc 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7045030 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:24.265 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x35ed483 0x35fc124 0x35fc215 0x3618092 0x9e33da7 0x9e33e69 0x9ef3c57 0x9ef3dec 0x9ef417e 0x9ef5765 0x9ef6a80 0x9ef6ea0 0x9f1568c 0x9f18413 0x9ed89e1 0x9ed79e1 0x7f64059 0x7fb93c0 0x9121733 0x91240a1 0x7857565 0x7a67c92 0x704540f 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.265 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc58e /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0xa2ca8f3 0xa2cb0b5 0xa2cb6f2 0xa2f0155 0xa2f0c19 0xa2f0cc7 0xa2f0d75 0xa2f0e24 0xa2f0f06 0xa2f11e4 0xa2f20d8 0xa2f3a1d 0x9d7d31e 0x9d7d827 0x9d7d94b 0x9d7d9f9 0x9d7daa7 0x9d98a72 0x9d98b54 0x9daeb9c 0x9daed6f 0xcf9390 0x9d58869 0x9d801fd 0x9d81397 0x9d9ac19 0x9d9b8f0 0x9d9889b 0x7834654 0x78decea 0x78e3e6c 0x9603d12 0x96493fe 0x7fb64cb 0x7fb6a69 0x7fb6dc1 0x7fb73ea 0x7fb975d 0x9121733 0x91240a1 0x7857565 0x7a67c92 0x704540f 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.289 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a15245 0x2a69674 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:24.298 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.330 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.330 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.381 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 33 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0xcca009 0x10a279e 0xf47938 0x9e2678b 0x9ef7155 0x9ef7fcf 0x9ef84cd 0x9ef87c0 0x9dd834b 0x9dd8586 0x9dd887d 0x9dd88fa 0x7b8b707 0x7b8bdd2 0x790e03f 0x790e27a 0x7a50f0f 0x7a510c8 0x725af59 0x7262199 0x70158ec 0x74fcb1a 0x74fcc6e 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.454 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x3617fd7 0x9e33da7 0x9e33ed4 0x9ef3c57 0x9ef3dec 0x9ef417e 0x9ef5765 0x9ef6a80 0x9ef6ea0 0x9f1372b 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.454 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x31826f9 0x9e21d5d 0x9e220d2 0x9e22b81 0x9e2307f 0x9e236af 0x9e23819 0x9f03b89 0x9f1698c 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.454 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6583 /lib64/libasan.so.8+0x15197 0x11380c8 0xf4792d 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e22e1c 0x9e2ae9f 0x9e2b21b 0x9f13d68 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.536 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x7d56cd6 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:24.552 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:24.552 INFO:teuthology.orchestra.run.trial158.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ebc /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:26.725 INFO:teuthology.orchestra.run.trial158.stderr:----------------------------------------------------- 2026-02-25T08:40:26.726 INFO:teuthology.orchestra.run.trial158.stderr:Suppressions used: 2026-02-25T08:40:26.726 INFO:teuthology.orchestra.run.trial158.stderr: count bytes template 2026-02-25T08:40:26.726 INFO:teuthology.orchestra.run.trial158.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:26.726 INFO:teuthology.orchestra.run.trial158.stderr:----------------------------------------------------- 2026-02-25T08:40:26.726 INFO:teuthology.orchestra.run.trial158.stderr: 2026-02-25T08:40:26.760 DEBUG:teuthology.orchestra.run.trial158:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-25T08:40:26.786 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-25T08:40:26.786 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:40:26.786 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-25T08:40:26.813 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:40:26.813 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-25T08:40:26.877 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:40:26.877 DEBUG:teuthology.orchestra.run.trial158:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-25T08:40:26.903 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:40:26.903 DEBUG:teuthology.orchestra.run.trial158:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-25T08:40:26.967 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:40:26.967 DEBUG:teuthology.orchestra.run.trial158:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-25T08:40:27.030 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:40:27.030 DEBUG:teuthology.orchestra.run.trial006:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-25T08:40:27.048 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:40:27.048 DEBUG:teuthology.orchestra.run.trial158:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2026-02-25T08:40:27.086 INFO:tasks.ceph:Adding keys to all mons... 2026-02-25T08:40:27.087 DEBUG:teuthology.orchestra.run.trial006:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-25T08:40:27.110 INFO:teuthology.orchestra.run.trial006.stdout:[osd.0] 2026-02-25T08:40:27.110 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBntZ5pBJoYABAAnMsBTYAfmzmMiffO9q7LgQ== 2026-02-25T08:40:27.110 INFO:teuthology.orchestra.run.trial006.stdout:[osd.1] 2026-02-25T08:40:27.110 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBstZ5p/0GgChAATc0bi4qkAeQ++IZYq5+VfQ== 2026-02-25T08:40:27.111 INFO:teuthology.orchestra.run.trial006.stdout:[mgr.x] 2026-02-25T08:40:27.111 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBitZ5p+pBgLhAAPeHBbfE9KToXiqea+i23Kw== 2026-02-25T08:40:27.111 INFO:teuthology.orchestra.run.trial006.stdout:[osd.2] 2026-02-25T08:40:27.111 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBytZ5pStEvOBAAdimxzS4VLjxTAo5h4vL0BA== 2026-02-25T08:40:27.111 INFO:teuthology.orchestra.run.trial006.stdout:[osd.3] 2026-02-25T08:40:27.111 INFO:teuthology.orchestra.run.trial006.stdout: key = AQB4tZ5pz9WRBhAAu0193nLSYuMS+vA0I4iIhg== 2026-02-25T08:40:27.111 INFO:teuthology.orchestra.run.trial006.stdout:[client.0] 2026-02-25T08:40:27.111 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBitZ5pFTU3MRAAqqggW7ULt6Ex4MglSUABfw== 2026-02-25T08:40:27.111 INFO:teuthology.orchestra.run.trial006.stdout:[client.1] 2026-02-25T08:40:27.111 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBitZ5pjNs1NBAA9CEDvz52jg0eRexKUPCKgw== 2026-02-25T08:40:27.113 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T08:40:27.201 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T08:40:27.250 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-25T08:40:27.296 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T08:40:27.345 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T08:40:27.394 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-25T08:40:27.443 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-25T08:40:27.493 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-25T08:40:27.494 DEBUG:teuthology.orchestra.run.trial006:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-25T08:40:27.519 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-25T08:40:27.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:27.603+0000 7fb05c038c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:27.603+0000 7fb05c038c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.616 DEBUG:teuthology.orchestra.run.trial006:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-25T08:40:27.641 DEBUG:teuthology.orchestra.run.trial006:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:27.697 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-25T08:40:27.697 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-25T08:40:27.697 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-25T08:40:27.739 INFO:tasks.ceph.mon.a:Started 2026-02-25T08:40:27.739 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-25T08:40:27.740 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-25T08:40:27.740 DEBUG:teuthology.orchestra.run.trial158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-25T08:40:27.742 INFO:tasks.ceph.mgr.x:Started 2026-02-25T08:40:27.743 DEBUG:tasks.ceph:set 0 configs 2026-02-25T08:40:27.743 DEBUG:teuthology.orchestra.run.trial006:> sudo ceph --cluster ceph config dump 2026-02-25T08:40:27.802 INFO:tasks.ceph.mon.a.trial006.stderr:2026-02-25T08:40:27.801+0000 7f599bc2fc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.803 INFO:tasks.ceph.mon.a.trial006.stderr:2026-02-25T08:40:27.801+0000 7f599bc2fc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.803 INFO:tasks.ceph.mon.a.trial006.stderr:2026-02-25T08:40:27.801+0000 7f599bc2fc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.803 INFO:tasks.ceph.mgr.x.trial158.stderr:2026-02-25T08:40:27.800+0000 7f6ddf036fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:27.813+0000 7f000ae3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:27.819+0000 7f000ae3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.823 INFO:tasks.ceph.mgr.x.trial158.stderr:2026-02-25T08:40:27.820+0000 7f6ddf036fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.824 INFO:tasks.ceph.mgr.x.trial158.stderr:2026-02-25T08:40:27.821+0000 7f6ddf036fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.852 INFO:teuthology.orchestra.run.trial006.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-25T08:40:27.860 INFO:tasks.ceph:Setting crush tunables to default 2026-02-25T08:40:27.861 DEBUG:teuthology.orchestra.run.trial006:> sudo ceph --cluster ceph osd crush tunables default 2026-02-25T08:40:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:27.930+0000 7fa3b7fa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:27.936+0000 7fa3b7fa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:27.971 INFO:teuthology.orchestra.run.trial006.stderr:adjusted tunables profile to default 2026-02-25T08:40:27.979 INFO:tasks.ceph:check_enable_crimson: True 2026-02-25T08:40:27.979 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-25T08:40:27.979 DEBUG:teuthology.orchestra.run.trial006:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-25T08:40:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:28.050+0000 7f54e89b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:28.056+0000 7f54e89b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:28.095 INFO:tasks.ceph.mon.a.trial006.stderr:2026-02-25T08:40:28.094+0000 7f5992ff1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-25T08:40:28.096 INFO:tasks.ceph.mon.a.trial006.stderr:Please be aware that this feature is experimental, untested, 2026-02-25T08:40:28.096 INFO:tasks.ceph.mon.a.trial006.stderr:unsupported, and may result in data corruption, data loss, 2026-02-25T08:40:28.096 INFO:tasks.ceph.mon.a.trial006.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-25T08:40:28.096 INFO:tasks.ceph.mon.a.trial006.stderr:feature with important data. 2026-02-25T08:40:28.096 INFO:tasks.ceph.mon.a.trial006.stderr: 2026-02-25T08:40:28.658 INFO:tasks.ceph.mgr.x.trial158.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-25T08:40:28.658 INFO:tasks.ceph.mgr.x.trial158.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-25T08:40:28.658 INFO:tasks.ceph.mgr.x.trial158.stderr: from numpy import show_config as show_numpy_config 2026-02-25T08:40:28.984 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-25T08:40:28.985 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:40:28.985 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-25T08:40:29.010 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-25T08:40:29.010 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-25T08:40:29.074 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:40:29.074 DEBUG:teuthology.orchestra.run.trial158:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-25T08:40:29.099 DEBUG:teuthology.orchestra.run.trial158:> set -ex 2026-02-25T08:40:29.099 DEBUG:teuthology.orchestra.run.trial158:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-25T08:40:29.165 DEBUG:teuthology.orchestra.run.trial158:> sudo ceph --cluster ceph osd new 95a533b1-c7bb-4fce-9178-fa2c372f3c68 0 2026-02-25T08:40:29.279 INFO:teuthology.orchestra.run.trial158.stderr:2026-02-25T08:40:29.273+0000 7f93eab83640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:29.281 INFO:teuthology.orchestra.run.trial158.stderr:2026-02-25T08:40:29.279+0000 7f93eab83640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:29.320 INFO:teuthology.orchestra.run.trial158.stdout:0 2026-02-25T08:40:29.328 DEBUG:teuthology.orchestra.run.trial158:> sudo ceph --cluster ceph osd new 906b4c51-50a4-4aa6-9e6c-67d7ce63f58f 1 2026-02-25T08:40:29.403 INFO:teuthology.orchestra.run.trial158.stderr:2026-02-25T08:40:29.397+0000 7f623e451640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:29.406 INFO:teuthology.orchestra.run.trial158.stderr:2026-02-25T08:40:29.402+0000 7f623e451640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:29.443 INFO:teuthology.orchestra.run.trial158.stdout:1 2026-02-25T08:40:29.453 DEBUG:teuthology.orchestra.run.trial158:> sudo ceph --cluster ceph osd new 6177afdc-ca29-4995-813c-7da15e1a8a79 2 2026-02-25T08:40:29.525 INFO:teuthology.orchestra.run.trial158.stderr:2026-02-25T08:40:29.519+0000 7effaa2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:29.528 INFO:teuthology.orchestra.run.trial158.stderr:2026-02-25T08:40:29.525+0000 7effaa2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:29.566 INFO:teuthology.orchestra.run.trial158.stdout:2 2026-02-25T08:40:29.575 DEBUG:teuthology.orchestra.run.trial158:> sudo ceph --cluster ceph osd new 2bacc44a-e8c0-4d21-a367-63f37ee84a95 3 2026-02-25T08:40:29.650 INFO:teuthology.orchestra.run.trial158.stderr:2026-02-25T08:40:29.643+0000 7f68c7406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:29.652 INFO:teuthology.orchestra.run.trial158.stderr:2026-02-25T08:40:29.649+0000 7f68c7406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:29.690 INFO:teuthology.orchestra.run.trial158.stdout:3 2026-02-25T08:40:29.698 DEBUG:teuthology.orchestra.run.trial006:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-25T08:40:29.727 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-25T08:40:29.728 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-25T08:40:29.769 INFO:tasks.ceph.osd.0:Started 2026-02-25T08:40:29.769 DEBUG:teuthology.orchestra.run.trial006:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-25T08:40:29.798 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-25T08:40:29.799 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-25T08:40:29.841 INFO:tasks.ceph.osd.1:Started 2026-02-25T08:40:29.841 DEBUG:teuthology.orchestra.run.trial158:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-25T08:40:29.868 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-25T08:40:29.868 DEBUG:teuthology.orchestra.run.trial158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-25T08:40:29.911 INFO:tasks.ceph.osd.2:Started 2026-02-25T08:40:29.911 DEBUG:teuthology.orchestra.run.trial158:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2026-02-25T08:40:29.941 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-25T08:40:29.941 DEBUG:teuthology.orchestra.run.trial158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2026-02-25T08:40:29.984 INFO:tasks.ceph.osd.3:Started 2026-02-25T08:40:29.985 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T08:40:30.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:30.058+0000 7f025b90b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:30.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:30.062+0000 7f025b90b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:30.096 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:30.096 INFO:teuthology.orchestra.run.trial006.stdout:{"epoch":7,"fsid":"b1b76914-ff72-4771-8729-7fc8b1bbfc83","created":"2026-02-25T08:40:27.813218+0000","modified":"2026-02-25T08:40:29.689334+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T08:40:29.689334+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"95a533b1-c7bb-4fce-9178-fa2c372f3c68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"906b4c51-50a4-4aa6-9e6c-67d7ce63f58f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6177afdc-ca29-4995-813c-7da15e1a8a79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2bacc44a-e8c0-4d21-a367-63f37ee84a95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:40:30.105 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-25T08:40:30.105 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-25T08:40:30.509 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T08:40:30.576 INFO:teuthology.misc.health.trial006.stderr:2026-02-25T08:40:30.573+0000 7fc6514c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:30.579 INFO:teuthology.misc.health.trial006.stderr:2026-02-25T08:40:30.578+0000 7fc6514c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:30.612 INFO:teuthology.misc.health.trial006.stdout: 2026-02-25T08:40:30.613 INFO:teuthology.misc.health.trial006.stdout:{"epoch":7,"fsid":"b1b76914-ff72-4771-8729-7fc8b1bbfc83","created":"2026-02-25T08:40:27.813218+0000","modified":"2026-02-25T08:40:29.689334+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T08:40:29.689334+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"95a533b1-c7bb-4fce-9178-fa2c372f3c68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"906b4c51-50a4-4aa6-9e6c-67d7ce63f58f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6177afdc-ca29-4995-813c-7da15e1a8a79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2bacc44a-e8c0-4d21-a367-63f37ee84a95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:40:30.619 DEBUG:teuthology.misc:0 of 4 OSDs are up 2026-02-25T08:40:31.004 INFO:tasks.ceph.mgr.x.trial158.stderr:2026-02-25T08:40:31.001+0000 7f6cfa45f640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-25T08:40:37.023 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T08:40:37.090 INFO:teuthology.misc.health.trial006.stderr:2026-02-25T08:40:37.086+0000 7fad2df0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:37.093 INFO:teuthology.misc.health.trial006.stderr:2026-02-25T08:40:37.092+0000 7fad2df0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:37.203 INFO:teuthology.misc.health.trial006.stdout: 2026-02-25T08:40:37.203 INFO:teuthology.misc.health.trial006.stdout:{"epoch":11,"fsid":"b1b76914-ff72-4771-8729-7fc8b1bbfc83","created":"2026-02-25T08:40:27.813218+0000","modified":"2026-02-25T08:40:37.013887+0000","last_up_change":"2026-02-25T08:40:37.013887+0000","last_in_change":"2026-02-25T08:40:29.689334+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"95a533b1-c7bb-4fce-9178-fa2c372f3c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6807","nonce":3323702864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6808","nonce":3323702864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6805","nonce":3323702864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6804","nonce":3323702864}]},"public_addr":"10.20.193.6:6807/3323702864","cluster_addr":"10.20.193.6:6808/3323702864","heartbeat_back_addr":"10.20.193.6:6805/3323702864","heartbeat_front_addr":"10.20.193.6:6804/3323702864","state":["exists","up"]},{"osd":1,"uuid":"906b4c51-50a4-4aa6-9e6c-67d7ce63f58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6803","nonce":1330699648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6802","nonce":1330699648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6800","nonce":1330699648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6801","nonce":1330699648}]},"public_addr":"10.20.193.6:6803/1330699648","cluster_addr":"10.20.193.6:6802/1330699648","heartbeat_back_addr":"10.20.193.6:6800/1330699648","heartbeat_front_addr":"10.20.193.6:6801/1330699648","state":["exists","up"]},{"osd":2,"uuid":"6177afdc-ca29-4995-813c-7da15e1a8a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6802","nonce":4142340047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6803","nonce":4142340047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6806","nonce":4142340047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6807","nonce":4142340047}]},"public_addr":"10.20.193.158:6802/4142340047","cluster_addr":"10.20.193.158:6803/4142340047","heartbeat_back_addr":"10.20.193.158:6806/4142340047","heartbeat_front_addr":"10.20.193.158:6807/4142340047","state":["exists","up"]},{"osd":3,"uuid":"2bacc44a-e8c0-4d21-a367-63f37ee84a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6804","nonce":1801380841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6805","nonce":1801380841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6808","nonce":1801380841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6809","nonce":1801380841}]},"public_addr":"10.20.193.158:6804/1801380841","cluster_addr":"10.20.193.158:6805/1801380841","heartbeat_back_addr":"10.20.193.158:6808/1801380841","heartbeat_front_addr":"10.20.193.158:6809/1801380841","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:40:37.211 DEBUG:teuthology.misc:4 of 4 OSDs are up 2026-02-25T08:40:37.211 INFO:tasks.ceph:Creating RBD pool 2026-02-25T08:40:37.212 DEBUG:teuthology.orchestra.run.trial006:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-25T08:40:37.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:37.280+0000 7f4ab45b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:37.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:37.287+0000 7f4ab45b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:38.024 INFO:teuthology.orchestra.run.trial006.stderr:pool 'rbd' created 2026-02-25T08:40:38.036 DEBUG:teuthology.orchestra.run.trial006:> rbd --cluster ceph pool init rbd 2026-02-25T08:40:38.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:38.075+0000 7fdbb0410240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:38.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:38.075+0000 7fdbb0410240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:38.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:38.079+0000 7fdbb0410240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:41.058 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-25T08:40:41.059 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-25T08:40:41.059 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-25T08:40:41.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:41.171+0000 7fe2c4035640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:41.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:41.177+0000 7fe2c4035640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:41.287 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:41.297 INFO:teuthology.orchestra.run.trial006.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-25T08:40:41.297 INFO:tasks.ceph_manager:config epoch is 1 2026-02-25T08:40:41.297 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-25T08:40:41.298 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-25T08:40:41.298 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-25T08:40:41.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:41.372+0000 7f3b0ba5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:41.377+0000 7f3b0ba5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:41.522 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:41.532 INFO:teuthology.orchestra.run.trial006.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6800","nonce":2961728841},{"type":"v1","addr":"10.20.193.158:6801","nonce":2961728841}]},"active_addr":"10.20.193.158:6801/2961728841","active_change":"2026-02-25T08:40:29.998140+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.158:0","nonce":527666922}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.158:0","nonce":1250036061}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.158:0","nonce":771832985}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.158:0","nonce":3693464867}]}]} 2026-02-25T08:40:41.536 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-25T08:40:41.536 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-25T08:40:41.537 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T08:40:41.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:41.609+0000 7f038819c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:41.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:41.614+0000 7f038819c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:41.728 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:41.728 INFO:teuthology.orchestra.run.trial006.stdout:{"epoch":15,"fsid":"b1b76914-ff72-4771-8729-7fc8b1bbfc83","created":"2026-02-25T08:40:27.813218+0000","modified":"2026-02-25T08:40:41.030237+0000","last_up_change":"2026-02-25T08:40:37.013887+0000","last_in_change":"2026-02-25T08:40:29.689334+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-25T08:40:37.402268+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2026-02-25T08:40:38.021073+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"95a533b1-c7bb-4fce-9178-fa2c372f3c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6807","nonce":3323702864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6808","nonce":3323702864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6805","nonce":3323702864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6804","nonce":3323702864}]},"public_addr":"10.20.193.6:6807/3323702864","cluster_addr":"10.20.193.6:6808/3323702864","heartbeat_back_addr":"10.20.193.6:6805/3323702864","heartbeat_front_addr":"10.20.193.6:6804/3323702864","state":["exists","up"]},{"osd":1,"uuid":"906b4c51-50a4-4aa6-9e6c-67d7ce63f58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6803","nonce":1330699648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6802","nonce":1330699648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6800","nonce":1330699648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6801","nonce":1330699648}]},"public_addr":"10.20.193.6:6803/1330699648","cluster_addr":"10.20.193.6:6802/1330699648","heartbeat_back_addr":"10.20.193.6:6800/1330699648","heartbeat_front_addr":"10.20.193.6:6801/1330699648","state":["exists","up"]},{"osd":2,"uuid":"6177afdc-ca29-4995-813c-7da15e1a8a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6802","nonce":4142340047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6803","nonce":4142340047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6806","nonce":4142340047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6807","nonce":4142340047}]},"public_addr":"10.20.193.158:6802/4142340047","cluster_addr":"10.20.193.158:6803/4142340047","heartbeat_back_addr":"10.20.193.158:6806/4142340047","heartbeat_front_addr":"10.20.193.158:6807/4142340047","state":["exists","up"]},{"osd":3,"uuid":"2bacc44a-e8c0-4d21-a367-63f37ee84a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6804","nonce":1801380841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6805","nonce":1801380841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6808","nonce":1801380841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6809","nonce":1801380841}]},"public_addr":"10.20.193.158:6804/1801380841","cluster_addr":"10.20.193.158:6805/1801380841","heartbeat_back_addr":"10.20.193.158:6808/1801380841","heartbeat_front_addr":"10.20.193.158:6809/1801380841","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:40:41.737 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-25T08:40:41.738 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T08:40:41.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:41.809+0000 7f4a20fcc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:41.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:41.814+0000 7f4a20fcc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:41.926 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:41.927 INFO:teuthology.orchestra.run.trial006.stdout:{"epoch":15,"fsid":"b1b76914-ff72-4771-8729-7fc8b1bbfc83","created":"2026-02-25T08:40:27.813218+0000","modified":"2026-02-25T08:40:41.030237+0000","last_up_change":"2026-02-25T08:40:37.013887+0000","last_in_change":"2026-02-25T08:40:29.689334+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-25T08:40:37.402268+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2026-02-25T08:40:38.021073+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"95a533b1-c7bb-4fce-9178-fa2c372f3c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6807","nonce":3323702864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6808","nonce":3323702864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6805","nonce":3323702864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6804","nonce":3323702864}]},"public_addr":"10.20.193.6:6807/3323702864","cluster_addr":"10.20.193.6:6808/3323702864","heartbeat_back_addr":"10.20.193.6:6805/3323702864","heartbeat_front_addr":"10.20.193.6:6804/3323702864","state":["exists","up"]},{"osd":1,"uuid":"906b4c51-50a4-4aa6-9e6c-67d7ce63f58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6803","nonce":1330699648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6802","nonce":1330699648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6800","nonce":1330699648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6801","nonce":1330699648}]},"public_addr":"10.20.193.6:6803/1330699648","cluster_addr":"10.20.193.6:6802/1330699648","heartbeat_back_addr":"10.20.193.6:6800/1330699648","heartbeat_front_addr":"10.20.193.6:6801/1330699648","state":["exists","up"]},{"osd":2,"uuid":"6177afdc-ca29-4995-813c-7da15e1a8a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6802","nonce":4142340047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6803","nonce":4142340047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6806","nonce":4142340047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6807","nonce":4142340047}]},"public_addr":"10.20.193.158:6802/4142340047","cluster_addr":"10.20.193.158:6803/4142340047","heartbeat_back_addr":"10.20.193.158:6806/4142340047","heartbeat_front_addr":"10.20.193.158:6807/4142340047","state":["exists","up"]},{"osd":3,"uuid":"2bacc44a-e8c0-4d21-a367-63f37ee84a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6804","nonce":1801380841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6805","nonce":1801380841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6808","nonce":1801380841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.158:6809","nonce":1801380841}]},"public_addr":"10.20.193.158:6804/1801380841","cluster_addr":"10.20.193.158:6805/1801380841","heartbeat_back_addr":"10.20.193.158:6808/1801380841","heartbeat_front_addr":"10.20.193.158:6809/1801380841","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:40:41.937 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-25T08:40:41.937 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-25T08:40:41.938 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-25T08:40:41.938 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-25T08:40:42.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.017+0000 7f758a446640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.022 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.018+0000 7f5b6a835640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.018+0000 7fd8d1dfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.018+0000 7f0b02cb7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.022+0000 7f758a446640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.023+0000 7f5b6a835640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.026 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.024+0000 7fd8d1dfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.026 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.024+0000 7f0b02cb7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.044 INFO:teuthology.orchestra.run.trial006.stdout:42949672966 2026-02-25T08:40:42.044 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-25T08:40:42.045 INFO:teuthology.orchestra.run.trial006.stdout:47244640261 2026-02-25T08:40:42.046 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-25T08:40:42.047 INFO:teuthology.orchestra.run.trial006.stdout:47244640261 2026-02-25T08:40:42.047 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-25T08:40:42.049 INFO:teuthology.orchestra.run.trial006.stdout:42949672966 2026-02-25T08:40:42.050 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-25T08:40:42.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.121+0000 7f74a0962640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.127+0000 7f74a0962640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.166+0000 7ff209e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.167+0000 7f4918d8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.168+0000 7fd941c1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.172+0000 7ff209e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.173+0000 7f4918d8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.173+0000 7fd941c1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.243 INFO:teuthology.orchestra.run.trial006.stdout:42949672966 2026-02-25T08:40:42.254 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.2 2026-02-25T08:40:42.254 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:42.287 INFO:teuthology.orchestra.run.trial006.stdout:47244640261 2026-02-25T08:40:42.288 INFO:teuthology.orchestra.run.trial006.stdout:47244640261 2026-02-25T08:40:42.297 INFO:teuthology.orchestra.run.trial006.stdout:42949672966 2026-02-25T08:40:42.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.3 2026-02-25T08:40:42.298 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:42.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.0 2026-02-25T08:40:42.299 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:42.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.1 2026-02-25T08:40:42.307 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:42.308 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-25T08:40:42.308 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:40:42.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.419+0000 7f0b2a1ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.424+0000 7f0b2a1ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.536 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:42.537 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-25T08:40:42.545 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2026-02-25T08:40:42.036705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":4056,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208662568,"statfs":{"total":213674622976,"available":213670469632,"internally_reserved":0,"allocated":4153344,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.043008+0000","last_change":"2026-02-25T08:40:41.045744+0000","last_active":"2026-02-25T08:40:41.043008+0000","last_peered":"2026-02-25T08:40:41.043008+0000","last_clean":"2026-02-25T08:40:41.043008+0000","last_became_active":"2026-02-25T08:40:39.088696+0000","last_became_peered":"2026-02-25T08:40:39.088696+0000","last_unstale":"2026-02-25T08:40:41.043008+0000","last_undegraded":"2026-02-25T08:40:41.043008+0000","last_fullsized":"2026-02-25T08:40:41.043008+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.043049+0000","last_change":"2026-02-25T08:40:41.045214+0000","last_active":"2026-02-25T08:40:41.043049+0000","last_peered":"2026-02-25T08:40:41.043049+0000","last_clean":"2026-02-25T08:40:41.043049+0000","last_became_active":"2026-02-25T08:40:39.071192+0000","last_became_peered":"2026-02-25T08:40:39.071192+0000","last_unstale":"2026-02-25T08:40:41.043049+0000","last_undegraded":"2026-02-25T08:40:41.043049+0000","last_fullsized":"2026-02-25T08:40:41.043049+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040692+0000","last_change":"2026-02-25T08:40:41.042930+0000","last_active":"2026-02-25T08:40:41.040692+0000","last_peered":"2026-02-25T08:40:41.040692+0000","last_clean":"2026-02-25T08:40:41.040692+0000","last_became_active":"2026-02-25T08:40:39.066104+0000","last_became_peered":"2026-02-25T08:40:39.066104+0000","last_unstale":"2026-02-25T08:40:41.040692+0000","last_undegraded":"2026-02-25T08:40:41.040692+0000","last_fullsized":"2026-02-25T08:40:41.040692+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040718+0000","last_change":"2026-02-25T08:40:41.043204+0000","last_active":"2026-02-25T08:40:41.040718+0000","last_peered":"2026-02-25T08:40:41.040718+0000","last_clean":"2026-02-25T08:40:41.040718+0000","last_became_active":"2026-02-25T08:40:39.068832+0000","last_became_peered":"2026-02-25T08:40:39.068832+0000","last_unstale":"2026-02-25T08:40:41.040718+0000","last_undegraded":"2026-02-25T08:40:41.040718+0000","last_fullsized":"2026-02-25T08:40:41.040718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":82,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.042450+0000","last_change":"2026-02-25T08:40:39.100603+0000","last_active":"2026-02-25T08:40:41.042450+0000","last_peered":"2026-02-25T08:40:41.042450+0000","last_clean":"2026-02-25T08:40:41.042450+0000","last_became_active":"2026-02-25T08:40:39.090744+0000","last_became_peered":"2026-02-25T08:40:39.090744+0000","last_unstale":"2026-02-25T08:40:41.042450+0000","last_undegraded":"2026-02-25T08:40:41.042450+0000","last_fullsized":"2026-02-25T08:40:41.042450+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.041023+0000","last_change":"2026-02-25T08:40:41.043569+0000","last_active":"2026-02-25T08:40:41.041023+0000","last_peered":"2026-02-25T08:40:41.041023+0000","last_clean":"2026-02-25T08:40:41.041023+0000","last_became_active":"2026-02-25T08:40:39.072385+0000","last_became_peered":"2026-02-25T08:40:39.072385+0000","last_unstale":"2026-02-25T08:40:41.041023+0000","last_undegraded":"2026-02-25T08:40:41.041023+0000","last_fullsized":"2026-02-25T08:40:41.041023+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.042296+0000","last_change":"2026-02-25T08:40:41.042296+0000","last_active":"2026-02-25T08:40:41.042296+0000","last_peered":"2026-02-25T08:40:41.042296+0000","last_clean":"2026-02-25T08:40:41.042296+0000","last_became_active":"2026-02-25T08:40:39.072423+0000","last_became_peered":"2026-02-25T08:40:39.072423+0000","last_unstale":"2026-02-25T08:40:41.042296+0000","last_undegraded":"2026-02-25T08:40:41.042296+0000","last_fullsized":"2026-02-25T08:40:41.042296+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040777+0000","last_change":"2026-02-25T08:40:41.043202+0000","last_active":"2026-02-25T08:40:41.040777+0000","last_peered":"2026-02-25T08:40:41.040777+0000","last_clean":"2026-02-25T08:40:41.040777+0000","last_became_active":"2026-02-25T08:40:39.068039+0000","last_became_peered":"2026-02-25T08:40:39.068039+0000","last_unstale":"2026-02-25T08:40:41.040777+0000","last_undegraded":"2026-02-25T08:40:41.040777+0000","last_fullsized":"2026-02-25T08:40:41.040777+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.039223+0000","last_change":"2026-02-25T08:40:41.042120+0000","last_active":"2026-02-25T08:40:41.039223+0000","last_peered":"2026-02-25T08:40:41.039223+0000","last_clean":"2026-02-25T08:40:41.039223+0000","last_became_active":"2026-02-25T08:40:39.085713+0000","last_became_peered":"2026-02-25T08:40:39.085713+0000","last_unstale":"2026-02-25T08:40:41.039223+0000","last_undegraded":"2026-02-25T08:40:41.039223+0000","last_fullsized":"2026-02-25T08:40:41.039223+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":213674622976,"available":213670469632,"internally_reserved":0,"allocated":4153344,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":448,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166208,"statfs":{"total":53418655744,"available":53418196992,"internally_reserved":0,"allocated":458752,"data_stored":458752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1204,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165452,"statfs":{"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1204,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165452,"statfs":{"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1200,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165456,"statfs":{"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53418196992,"internally_reserved":0,"allocated":458752,"data_stored":458752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:40:42.547 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:40:42.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.619+0000 7f8b3ac1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.624+0000 7f8b3ac1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.734 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:42.734 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-25T08:40:42.743 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2026-02-25T08:40:42.036705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":4056,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208662568,"statfs":{"total":213674622976,"available":213670469632,"internally_reserved":0,"allocated":4153344,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.043008+0000","last_change":"2026-02-25T08:40:41.045744+0000","last_active":"2026-02-25T08:40:41.043008+0000","last_peered":"2026-02-25T08:40:41.043008+0000","last_clean":"2026-02-25T08:40:41.043008+0000","last_became_active":"2026-02-25T08:40:39.088696+0000","last_became_peered":"2026-02-25T08:40:39.088696+0000","last_unstale":"2026-02-25T08:40:41.043008+0000","last_undegraded":"2026-02-25T08:40:41.043008+0000","last_fullsized":"2026-02-25T08:40:41.043008+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.043049+0000","last_change":"2026-02-25T08:40:41.045214+0000","last_active":"2026-02-25T08:40:41.043049+0000","last_peered":"2026-02-25T08:40:41.043049+0000","last_clean":"2026-02-25T08:40:41.043049+0000","last_became_active":"2026-02-25T08:40:39.071192+0000","last_became_peered":"2026-02-25T08:40:39.071192+0000","last_unstale":"2026-02-25T08:40:41.043049+0000","last_undegraded":"2026-02-25T08:40:41.043049+0000","last_fullsized":"2026-02-25T08:40:41.043049+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040692+0000","last_change":"2026-02-25T08:40:41.042930+0000","last_active":"2026-02-25T08:40:41.040692+0000","last_peered":"2026-02-25T08:40:41.040692+0000","last_clean":"2026-02-25T08:40:41.040692+0000","last_became_active":"2026-02-25T08:40:39.066104+0000","last_became_peered":"2026-02-25T08:40:39.066104+0000","last_unstale":"2026-02-25T08:40:41.040692+0000","last_undegraded":"2026-02-25T08:40:41.040692+0000","last_fullsized":"2026-02-25T08:40:41.040692+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040718+0000","last_change":"2026-02-25T08:40:41.043204+0000","last_active":"2026-02-25T08:40:41.040718+0000","last_peered":"2026-02-25T08:40:41.040718+0000","last_clean":"2026-02-25T08:40:41.040718+0000","last_became_active":"2026-02-25T08:40:39.068832+0000","last_became_peered":"2026-02-25T08:40:39.068832+0000","last_unstale":"2026-02-25T08:40:41.040718+0000","last_undegraded":"2026-02-25T08:40:41.040718+0000","last_fullsized":"2026-02-25T08:40:41.040718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":82,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.042450+0000","last_change":"2026-02-25T08:40:39.100603+0000","last_active":"2026-02-25T08:40:41.042450+0000","last_peered":"2026-02-25T08:40:41.042450+0000","last_clean":"2026-02-25T08:40:41.042450+0000","last_became_active":"2026-02-25T08:40:39.090744+0000","last_became_peered":"2026-02-25T08:40:39.090744+0000","last_unstale":"2026-02-25T08:40:41.042450+0000","last_undegraded":"2026-02-25T08:40:41.042450+0000","last_fullsized":"2026-02-25T08:40:41.042450+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.041023+0000","last_change":"2026-02-25T08:40:41.043569+0000","last_active":"2026-02-25T08:40:41.041023+0000","last_peered":"2026-02-25T08:40:41.041023+0000","last_clean":"2026-02-25T08:40:41.041023+0000","last_became_active":"2026-02-25T08:40:39.072385+0000","last_became_peered":"2026-02-25T08:40:39.072385+0000","last_unstale":"2026-02-25T08:40:41.041023+0000","last_undegraded":"2026-02-25T08:40:41.041023+0000","last_fullsized":"2026-02-25T08:40:41.041023+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.042296+0000","last_change":"2026-02-25T08:40:41.042296+0000","last_active":"2026-02-25T08:40:41.042296+0000","last_peered":"2026-02-25T08:40:41.042296+0000","last_clean":"2026-02-25T08:40:41.042296+0000","last_became_active":"2026-02-25T08:40:39.072423+0000","last_became_peered":"2026-02-25T08:40:39.072423+0000","last_unstale":"2026-02-25T08:40:41.042296+0000","last_undegraded":"2026-02-25T08:40:41.042296+0000","last_fullsized":"2026-02-25T08:40:41.042296+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040777+0000","last_change":"2026-02-25T08:40:41.043202+0000","last_active":"2026-02-25T08:40:41.040777+0000","last_peered":"2026-02-25T08:40:41.040777+0000","last_clean":"2026-02-25T08:40:41.040777+0000","last_became_active":"2026-02-25T08:40:39.068039+0000","last_became_peered":"2026-02-25T08:40:39.068039+0000","last_unstale":"2026-02-25T08:40:41.040777+0000","last_undegraded":"2026-02-25T08:40:41.040777+0000","last_fullsized":"2026-02-25T08:40:41.040777+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.039223+0000","last_change":"2026-02-25T08:40:41.042120+0000","last_active":"2026-02-25T08:40:41.039223+0000","last_peered":"2026-02-25T08:40:41.039223+0000","last_clean":"2026-02-25T08:40:41.039223+0000","last_became_active":"2026-02-25T08:40:39.085713+0000","last_became_peered":"2026-02-25T08:40:39.085713+0000","last_unstale":"2026-02-25T08:40:41.039223+0000","last_undegraded":"2026-02-25T08:40:41.039223+0000","last_fullsized":"2026-02-25T08:40:41.039223+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":213674622976,"available":213670469632,"internally_reserved":0,"allocated":4153344,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":448,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166208,"statfs":{"total":53418655744,"available":53418196992,"internally_reserved":0,"allocated":458752,"data_stored":458752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1204,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165452,"statfs":{"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1204,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165452,"statfs":{"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1200,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165456,"statfs":{"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53418196992,"internally_reserved":0,"allocated":458752,"data_stored":458752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:40:42.744 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-25T08:40:42.744 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-25T08:40:42.744 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-25T08:40:42.745 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-25T08:40:42.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.815+0000 7fc3dc806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:42.819+0000 7fc3dc806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:42.959 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:42.959 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-25T08:40:42.969 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-25T08:40:42.969 INFO:teuthology.run_tasks:Running task thrashosds... 2026-02-25T08:40:42.986 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'chance_trim_stale_osdmaps': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2026-02-25T08:40:42.986 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-logstoremanager'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd target transaction size': 5, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'Degraded data redundancy', 'is down', 'osds down', 'pg .*? is .*?degraded.*?, acting', 'pg .*? is stuck', 'pg degraded', 'PG_BACKFILL_FULL', 'Low space hindering backfill .*? backfill_toofull', 'OSD_ROOT_DOWN', 'pgs degraded', 'pgs undersized', 'is active.*backfill_toofull.*'], 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'c32469671c6b0222f786dd36c34eb702f780a11d'}} 2026-02-25T08:40:42.987 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'chance_trim_stale_osdmaps': 0, 'dump_ops_enable': False, 'min_in': 4, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2026-02-25T08:40:42.987 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2026-02-25T08:40:43.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:43.058+0000 7f6ada55b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:43.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:43.065+0000 7f6ada55b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:43.189 INFO:tasks.thrashosds:Beginning thrashosds... 2026-02-25T08:40:43.189 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-25T08:40:43.189 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:40:43.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:43.262+0000 7f527fc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:43.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:43.267+0000 7f527fc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:43.378 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:43.378 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-25T08:40:43.387 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2026-02-25T08:40:42.036705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":4056,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208662568,"statfs":{"total":213674622976,"available":213670469632,"internally_reserved":0,"allocated":4153344,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.043008+0000","last_change":"2026-02-25T08:40:41.045744+0000","last_active":"2026-02-25T08:40:41.043008+0000","last_peered":"2026-02-25T08:40:41.043008+0000","last_clean":"2026-02-25T08:40:41.043008+0000","last_became_active":"2026-02-25T08:40:39.088696+0000","last_became_peered":"2026-02-25T08:40:39.088696+0000","last_unstale":"2026-02-25T08:40:41.043008+0000","last_undegraded":"2026-02-25T08:40:41.043008+0000","last_fullsized":"2026-02-25T08:40:41.043008+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.043049+0000","last_change":"2026-02-25T08:40:41.045214+0000","last_active":"2026-02-25T08:40:41.043049+0000","last_peered":"2026-02-25T08:40:41.043049+0000","last_clean":"2026-02-25T08:40:41.043049+0000","last_became_active":"2026-02-25T08:40:39.071192+0000","last_became_peered":"2026-02-25T08:40:39.071192+0000","last_unstale":"2026-02-25T08:40:41.043049+0000","last_undegraded":"2026-02-25T08:40:41.043049+0000","last_fullsized":"2026-02-25T08:40:41.043049+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040692+0000","last_change":"2026-02-25T08:40:41.042930+0000","last_active":"2026-02-25T08:40:41.040692+0000","last_peered":"2026-02-25T08:40:41.040692+0000","last_clean":"2026-02-25T08:40:41.040692+0000","last_became_active":"2026-02-25T08:40:39.066104+0000","last_became_peered":"2026-02-25T08:40:39.066104+0000","last_unstale":"2026-02-25T08:40:41.040692+0000","last_undegraded":"2026-02-25T08:40:41.040692+0000","last_fullsized":"2026-02-25T08:40:41.040692+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040718+0000","last_change":"2026-02-25T08:40:41.043204+0000","last_active":"2026-02-25T08:40:41.040718+0000","last_peered":"2026-02-25T08:40:41.040718+0000","last_clean":"2026-02-25T08:40:41.040718+0000","last_became_active":"2026-02-25T08:40:39.068832+0000","last_became_peered":"2026-02-25T08:40:39.068832+0000","last_unstale":"2026-02-25T08:40:41.040718+0000","last_undegraded":"2026-02-25T08:40:41.040718+0000","last_fullsized":"2026-02-25T08:40:41.040718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":82,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.042450+0000","last_change":"2026-02-25T08:40:39.100603+0000","last_active":"2026-02-25T08:40:41.042450+0000","last_peered":"2026-02-25T08:40:41.042450+0000","last_clean":"2026-02-25T08:40:41.042450+0000","last_became_active":"2026-02-25T08:40:39.090744+0000","last_became_peered":"2026-02-25T08:40:39.090744+0000","last_unstale":"2026-02-25T08:40:41.042450+0000","last_undegraded":"2026-02-25T08:40:41.042450+0000","last_fullsized":"2026-02-25T08:40:41.042450+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.041023+0000","last_change":"2026-02-25T08:40:41.043569+0000","last_active":"2026-02-25T08:40:41.041023+0000","last_peered":"2026-02-25T08:40:41.041023+0000","last_clean":"2026-02-25T08:40:41.041023+0000","last_became_active":"2026-02-25T08:40:39.072385+0000","last_became_peered":"2026-02-25T08:40:39.072385+0000","last_unstale":"2026-02-25T08:40:41.041023+0000","last_undegraded":"2026-02-25T08:40:41.041023+0000","last_fullsized":"2026-02-25T08:40:41.041023+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.042296+0000","last_change":"2026-02-25T08:40:41.042296+0000","last_active":"2026-02-25T08:40:41.042296+0000","last_peered":"2026-02-25T08:40:41.042296+0000","last_clean":"2026-02-25T08:40:41.042296+0000","last_became_active":"2026-02-25T08:40:39.072423+0000","last_became_peered":"2026-02-25T08:40:39.072423+0000","last_unstale":"2026-02-25T08:40:41.042296+0000","last_undegraded":"2026-02-25T08:40:41.042296+0000","last_fullsized":"2026-02-25T08:40:41.042296+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040777+0000","last_change":"2026-02-25T08:40:41.043202+0000","last_active":"2026-02-25T08:40:41.040777+0000","last_peered":"2026-02-25T08:40:41.040777+0000","last_clean":"2026-02-25T08:40:41.040777+0000","last_became_active":"2026-02-25T08:40:39.068039+0000","last_became_peered":"2026-02-25T08:40:39.068039+0000","last_unstale":"2026-02-25T08:40:41.040777+0000","last_undegraded":"2026-02-25T08:40:41.040777+0000","last_fullsized":"2026-02-25T08:40:41.040777+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.039223+0000","last_change":"2026-02-25T08:40:41.042120+0000","last_active":"2026-02-25T08:40:41.039223+0000","last_peered":"2026-02-25T08:40:41.039223+0000","last_clean":"2026-02-25T08:40:41.039223+0000","last_became_active":"2026-02-25T08:40:39.085713+0000","last_became_peered":"2026-02-25T08:40:39.085713+0000","last_unstale":"2026-02-25T08:40:41.039223+0000","last_undegraded":"2026-02-25T08:40:41.039223+0000","last_fullsized":"2026-02-25T08:40:41.039223+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":213674622976,"available":213670469632,"internally_reserved":0,"allocated":4153344,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":448,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166208,"statfs":{"total":53418655744,"available":53418196992,"internally_reserved":0,"allocated":458752,"data_stored":458752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1204,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165452,"statfs":{"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1204,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165452,"statfs":{"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1200,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165456,"statfs":{"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53418196992,"internally_reserved":0,"allocated":458752,"data_stored":458752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:40:43.388 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:40:43.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:43.462+0000 7f098810b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:43.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:43.467+0000 7f098810b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:43.584 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:43.584 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-25T08:40:43.593 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2026-02-25T08:40:42.036705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":4056,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208662568,"statfs":{"total":213674622976,"available":213670469632,"internally_reserved":0,"allocated":4153344,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.043008+0000","last_change":"2026-02-25T08:40:41.045744+0000","last_active":"2026-02-25T08:40:41.043008+0000","last_peered":"2026-02-25T08:40:41.043008+0000","last_clean":"2026-02-25T08:40:41.043008+0000","last_became_active":"2026-02-25T08:40:39.088696+0000","last_became_peered":"2026-02-25T08:40:39.088696+0000","last_unstale":"2026-02-25T08:40:41.043008+0000","last_undegraded":"2026-02-25T08:40:41.043008+0000","last_fullsized":"2026-02-25T08:40:41.043008+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.043049+0000","last_change":"2026-02-25T08:40:41.045214+0000","last_active":"2026-02-25T08:40:41.043049+0000","last_peered":"2026-02-25T08:40:41.043049+0000","last_clean":"2026-02-25T08:40:41.043049+0000","last_became_active":"2026-02-25T08:40:39.071192+0000","last_became_peered":"2026-02-25T08:40:39.071192+0000","last_unstale":"2026-02-25T08:40:41.043049+0000","last_undegraded":"2026-02-25T08:40:41.043049+0000","last_fullsized":"2026-02-25T08:40:41.043049+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040692+0000","last_change":"2026-02-25T08:40:41.042930+0000","last_active":"2026-02-25T08:40:41.040692+0000","last_peered":"2026-02-25T08:40:41.040692+0000","last_clean":"2026-02-25T08:40:41.040692+0000","last_became_active":"2026-02-25T08:40:39.066104+0000","last_became_peered":"2026-02-25T08:40:39.066104+0000","last_unstale":"2026-02-25T08:40:41.040692+0000","last_undegraded":"2026-02-25T08:40:41.040692+0000","last_fullsized":"2026-02-25T08:40:41.040692+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040718+0000","last_change":"2026-02-25T08:40:41.043204+0000","last_active":"2026-02-25T08:40:41.040718+0000","last_peered":"2026-02-25T08:40:41.040718+0000","last_clean":"2026-02-25T08:40:41.040718+0000","last_became_active":"2026-02-25T08:40:39.068832+0000","last_became_peered":"2026-02-25T08:40:39.068832+0000","last_unstale":"2026-02-25T08:40:41.040718+0000","last_undegraded":"2026-02-25T08:40:41.040718+0000","last_fullsized":"2026-02-25T08:40:41.040718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":82,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.042450+0000","last_change":"2026-02-25T08:40:39.100603+0000","last_active":"2026-02-25T08:40:41.042450+0000","last_peered":"2026-02-25T08:40:41.042450+0000","last_clean":"2026-02-25T08:40:41.042450+0000","last_became_active":"2026-02-25T08:40:39.090744+0000","last_became_peered":"2026-02-25T08:40:39.090744+0000","last_unstale":"2026-02-25T08:40:41.042450+0000","last_undegraded":"2026-02-25T08:40:41.042450+0000","last_fullsized":"2026-02-25T08:40:41.042450+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.041023+0000","last_change":"2026-02-25T08:40:41.043569+0000","last_active":"2026-02-25T08:40:41.041023+0000","last_peered":"2026-02-25T08:40:41.041023+0000","last_clean":"2026-02-25T08:40:41.041023+0000","last_became_active":"2026-02-25T08:40:39.072385+0000","last_became_peered":"2026-02-25T08:40:39.072385+0000","last_unstale":"2026-02-25T08:40:41.041023+0000","last_undegraded":"2026-02-25T08:40:41.041023+0000","last_fullsized":"2026-02-25T08:40:41.041023+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.042296+0000","last_change":"2026-02-25T08:40:41.042296+0000","last_active":"2026-02-25T08:40:41.042296+0000","last_peered":"2026-02-25T08:40:41.042296+0000","last_clean":"2026-02-25T08:40:41.042296+0000","last_became_active":"2026-02-25T08:40:39.072423+0000","last_became_peered":"2026-02-25T08:40:39.072423+0000","last_unstale":"2026-02-25T08:40:41.042296+0000","last_undegraded":"2026-02-25T08:40:41.042296+0000","last_fullsized":"2026-02-25T08:40:41.042296+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040777+0000","last_change":"2026-02-25T08:40:41.043202+0000","last_active":"2026-02-25T08:40:41.040777+0000","last_peered":"2026-02-25T08:40:41.040777+0000","last_clean":"2026-02-25T08:40:41.040777+0000","last_became_active":"2026-02-25T08:40:39.068039+0000","last_became_peered":"2026-02-25T08:40:39.068039+0000","last_unstale":"2026-02-25T08:40:41.040777+0000","last_undegraded":"2026-02-25T08:40:41.040777+0000","last_fullsized":"2026-02-25T08:40:41.040777+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.039223+0000","last_change":"2026-02-25T08:40:41.042120+0000","last_active":"2026-02-25T08:40:41.039223+0000","last_peered":"2026-02-25T08:40:41.039223+0000","last_clean":"2026-02-25T08:40:41.039223+0000","last_became_active":"2026-02-25T08:40:39.085713+0000","last_became_peered":"2026-02-25T08:40:39.085713+0000","last_unstale":"2026-02-25T08:40:41.039223+0000","last_undegraded":"2026-02-25T08:40:41.039223+0000","last_fullsized":"2026-02-25T08:40:41.039223+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":213674622976,"available":213670469632,"internally_reserved":0,"allocated":4153344,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":448,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166208,"statfs":{"total":53418655744,"available":53418196992,"internally_reserved":0,"allocated":458752,"data_stored":458752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1204,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165452,"statfs":{"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1204,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165452,"statfs":{"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1200,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165456,"statfs":{"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53418196992,"internally_reserved":0,"allocated":458752,"data_stored":458752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:40:43.594 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-25T08:40:43.595 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2026-02-25T08:40:43.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:43.667+0000 7f2cd1774640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:43.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:43.672+0000 7f2cd1774640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:43.783 INFO:teuthology.orchestra.run.trial006.stdout:epoch 15 2026-02-25T08:40:43.783 INFO:teuthology.orchestra.run.trial006.stdout:fsid b1b76914-ff72-4771-8729-7fc8b1bbfc83 2026-02-25T08:40:43.783 INFO:teuthology.orchestra.run.trial006.stdout:created 2026-02-25T08:40:27.813218+0000 2026-02-25T08:40:43.783 INFO:teuthology.orchestra.run.trial006.stdout:modified 2026-02-25T08:40:41.030237+0000 2026-02-25T08:40:43.783 INFO:teuthology.orchestra.run.trial006.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2026-02-25T08:40:43.784 INFO:teuthology.orchestra.run.trial006.stdout:crush_version 5 2026-02-25T08:40:43.784 INFO:teuthology.orchestra.run.trial006.stdout:full_ratio 0.95 2026-02-25T08:40:43.784 INFO:teuthology.orchestra.run.trial006.stdout:backfillfull_ratio 0.9 2026-02-25T08:40:43.784 INFO:teuthology.orchestra.run.trial006.stdout:nearfull_ratio 0.85 2026-02-25T08:40:43.784 INFO:teuthology.orchestra.run.trial006.stdout:require_min_compat_client luminous 2026-02-25T08:40:43.784 INFO:teuthology.orchestra.run.trial006.stdout:min_compat_client jewel 2026-02-25T08:40:43.784 INFO:teuthology.orchestra.run.trial006.stdout:require_osd_release tentacle 2026-02-25T08:40:43.784 INFO:teuthology.orchestra.run.trial006.stdout:stretch_mode_enabled false 2026-02-25T08:40:43.784 INFO:teuthology.orchestra.run.trial006.stdout:allow_crimson=true 2026-02-25T08:40:43.784 INFO:teuthology.orchestra.run.trial006.stdout:pool 1 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 15 flags hashpspool,selfmanaged_snaps,crimson stripe_width 0 application rbd read_balance_score 1.50 2026-02-25T08:40:43.785 INFO:teuthology.orchestra.run.trial006.stdout: removed_snaps_queue [2~1] 2026-02-25T08:40:43.785 INFO:teuthology.orchestra.run.trial006.stdout:pool 2 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 14 flags hashpspool,crimson stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 4.00 2026-02-25T08:40:43.785 INFO:teuthology.orchestra.run.trial006.stdout:max_osd 4 2026-02-25T08:40:43.785 INFO:teuthology.orchestra.run.trial006.stdout:osd.0 up in weight 1 up_from 11 up_thru 12 down_at 0 last_clean_interval [0,0) v2:10.20.193.6:6807/3323702864 v2:10.20.193.6:6808/3323702864 exists,up 95a533b1-c7bb-4fce-9178-fa2c372f3c68 2026-02-25T08:40:43.785 INFO:teuthology.orchestra.run.trial006.stdout:osd.1 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v2:10.20.193.6:6803/1330699648 v2:10.20.193.6:6802/1330699648 exists,up 906b4c51-50a4-4aa6-9e6c-67d7ce63f58f 2026-02-25T08:40:43.785 INFO:teuthology.orchestra.run.trial006.stdout:osd.2 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v2:10.20.193.158:6802/4142340047 v2:10.20.193.158:6803/4142340047 exists,up 6177afdc-ca29-4995-813c-7da15e1a8a79 2026-02-25T08:40:43.785 INFO:teuthology.orchestra.run.trial006.stdout:osd.3 up in weight 1 up_from 11 up_thru 12 down_at 0 last_clean_interval [0,0) v2:10.20.193.158:6804/1801380841 v2:10.20.193.158:6805/1801380841 exists,up 2bacc44a-e8c0-4d21-a367-63f37ee84a95 2026-02-25T08:40:43.792 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 11 up_thru 12 down_at 0 last_clean_interval [0,0) v2:10.20.193.6:6807/3323702864 v2:10.20.193.6:6808/3323702864 exists,up 95a533b1-c7bb-4fce-9178-fa2c372f3c68', 'osd.1 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v2:10.20.193.6:6803/1330699648 v2:10.20.193.6:6802/1330699648 exists,up 906b4c51-50a4-4aa6-9e6c-67d7ce63f58f', 'osd.2 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v2:10.20.193.158:6802/4142340047 v2:10.20.193.158:6803/4142340047 exists,up 6177afdc-ca29-4995-813c-7da15e1a8a79', 'osd.3 up in weight 1 up_from 11 up_thru 12 down_at 0 last_clean_interval [0,0) v2:10.20.193.158:6804/1801380841 v2:10.20.193.158:6805/1801380841 exists,up 2bacc44a-e8c0-4d21-a367-63f37ee84a95'] 2026-02-25T08:40:43.793 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2026-02-25T08:40:43.867 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-25T08:40:43.867 INFO:teuthology.orchestra.run.trial006.stdout: "name": "mon.a", 2026-02-25T08:40:43.867 INFO:teuthology.orchestra.run.trial006.stdout: "cluster": "ceph", 2026-02-25T08:40:43.868 INFO:teuthology.orchestra.run.trial006.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2026-02-25T08:40:43.868 INFO:teuthology.orchestra.run.trial006.stdout: "admin_socket_mode": "", 2026-02-25T08:40:43.868 INFO:teuthology.orchestra.run.trial006.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2026-02-25T08:40:43.868 INFO:teuthology.orchestra.run.trial006.stdout: "auth_client_required": "cephx, none", 2026-02-25T08:40:43.868 INFO:teuthology.orchestra.run.trial006.stdout: "auth_cluster_required": "cephx", 2026-02-25T08:40:43.868 INFO:teuthology.orchestra.run.trial006.stdout: "auth_debug": "true", 2026-02-25T08:40:43.868 INFO:teuthology.orchestra.run.trial006.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2026-02-25T08:40:43.868 INFO:teuthology.orchestra.run.trial006.stdout: "auth_mon_ticket_ttl": "660.000000", 2026-02-25T08:40:43.868 INFO:teuthology.orchestra.run.trial006.stdout: "auth_service_required": "cephx", 2026-02-25T08:40:43.868 INFO:teuthology.orchestra.run.trial006.stdout: "auth_service_ticket_ttl": "240.000000", 2026-02-25T08:40:43.868 INFO:teuthology.orchestra.run.trial006.stdout: "auth_supported": "cephx", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_aio": "true", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_aio_max_queue_depth": "1024", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_aio_poll_ms": "250", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_aio_reap_max": "16", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_aio_submit_retry_initial_delay_us": "125", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_aio_submit_retry_max": "16", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_async_discard": "false", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_async_discard_max_pending": "1000000", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_async_discard_threads": "0", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_block_size": "4096", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_debug_aio": "false", 2026-02-25T08:40:43.869 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_debug_aio_log_age": "5.000000", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_debug_discard_sleep": "0", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_debug_inflight_ios": "false", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_discard_max_bytes": "10737418240", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_enable_discard": "false", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_flock_retry": "3", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_flock_retry_interval": "0.100000", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_inject_crash": "0", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_inject_crash_flush_delay": "2", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_ioring": "false", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_ioring_hipri": "false", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_ioring_sqthread_poll": "false", 2026-02-25T08:40:43.870 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_max_discard_length": "2147483648", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_nvme_unbind_from_kernel": "false", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_read_buffer_alignment": "4096", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_read_preallocated_huge_buffers": "", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_stalled_read_warn_threshold": "1", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bdev_type": "", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_alloc_size": "1048576", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_allocator": "hybrid", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_buffered_io": "true", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_check_for_zeros": "false", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_check_volume_selector_often": "false", 2026-02-25T08:40:43.871 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_check_volume_selector_on_umount": "false", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_compact_log_sync": "false", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_log_compact_min_size": "16777216", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_log_replay_check_allocations": "true", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_max_log_runway": "4194304", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_max_prefetch": "1048576", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_min_flush_size": "524288", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_min_log_runway": "1048576", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_replay_recovery": "false", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_replay_recovery_disable_compact": "false", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_shared_alloc_size": "65536", 2026-02-25T08:40:43.872 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_sync_write": "false", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluefs_wal_envelope_mode": "true", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_allocation_from_file": "true", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_allocator": "hybrid", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_allocator_lookup_policy": "auto", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_async_db_compaction": "true", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2026-02-25T08:40:43.873 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_bdev_label_multi": "true", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_bdev_label_require_all": "true", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_bitmapallocator_span_size": "1024", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_blobid_prealloc": "10240", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_block_create": "true", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_block_db_create": "false", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_block_db_path": "", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_block_db_size": "0", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_block_path": "", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_block_preallocate_file": "false", 2026-02-25T08:40:43.874 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_block_size": "107374182400", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_block_wal_create": "false", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_block_wal_path": "", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_block_wal_size": "100663296", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_bluefs": "true", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_bluefs_env_mirror": "false", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_bluefs_max_free": "10737418240", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2026-02-25T08:40:43.875 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_autotune": "true", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_autotune_interval": "5.000000", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_kv_ratio": "0.450000", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_meta_ratio": "0.450000", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_size": "0", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_size_hdd": "1073741824", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_size_ssd": "3221225472", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_trim_interval": "0.050000", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cache_type": "2q", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2026-02-25T08:40:43.876 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_clone_cow": "true", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_compression_algorithm": "snappy", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_compression_max_blob_size": "0", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_compression_min_blob_size": "0", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_compression_mode": "none", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_compression_required_ratio": "0.875000", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_csum_type": "crc32c", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_enforce_settings": "default", 2026-02-25T08:40:43.877 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_extent_map_encode_check": "false", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_freelist": "false", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_fsck_abort": "false", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_inject_read_err": "false", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_legacy_omap": "false", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_no_reuse_blocks": "false", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_omit_block_device_write": "false", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_omit_kv_commit": "false", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_onode_segmentation_random": "false", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_prefragment_max": "1048576", 2026-02-25T08:40:43.878 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_random_read_err": "0.000000", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_small_allocations": "0", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_default_buffered_read": "true", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_default_buffered_write": "false", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_deferred_batch_ops": "0", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_discard_on_mkfs": "true", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_elastic_shared_blobs": "true", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_extent_map_shard_max_size": "1200", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_extent_map_shard_min_size": "150", 2026-02-25T08:40:43.879 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_extent_map_shard_target_size": "500", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fail_eio": "false", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fragmentation_check_period": "3600", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_freelist_blocks_per_key": "128", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_on_mkfs": "true", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_on_mount": "false", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_on_mount_deep": "false", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_on_umount": "false", 2026-02-25T08:40:43.880 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_on_umount_deep": "false", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_quick_fix_threads": "2", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_gc_enable_blob_threshold": "0", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_gc_enable_total_threshold": "0", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_ignore_data_csum": "false", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_kvbackend": "rocksdb", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_log_collection_list_age": "60.000000", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2026-02-25T08:40:43.881 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_log_op_age": "5.000000", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_max_alloc_size": "0", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_max_blob_size": "0", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_max_blob_size_hdd": "65536", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_max_blob_size_ssd": "65536", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_max_defer_interval": "3.000000", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_max_deferred_txc": "32", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_min_alloc_size": "0", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_min_alloc_size_hdd": "4096", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_min_alloc_size_ssd": "4096", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_nid_prealloc": "1024", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_onode_segment_size": "0", 2026-02-25T08:40:43.882 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_prefer_deferred_size": "0", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_qfsck_on_mount": "true", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_recompression_min_gain": "1.200000", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_retry_disk_reads": "3", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_rocksdb_cf": "true", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_rocksdb_options_annex": "", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_slow_ops_warn_threshold": "1", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_spdk_coremask": "0x1", 2026-02-25T08:40:43.883 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_spdk_io_sleep": "5", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_spdk_max_io_completion": "0", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_spdk_mem": "512", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_sync_submit_transaction": "false", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_throttle_bytes": "67108864", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_throttle_cost_per_io": "0", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_throttle_trace_rate": "0.000000", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_tracing": "false", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2026-02-25T08:40:43.884 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_volume_selection_reserved": "0", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_warn_on_free_fragmentation": "0.800000", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_warn_on_legacy_statfs": "true", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_write_v2": "false", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_write_v2_random": "false", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "bluestore_zero_block_detection": "false", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "breakpad": "false", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "ceph_assert_supresssions": "", 2026-02-25T08:40:43.885 INFO:teuthology.orchestra.run.trial006.stdout: "cephadm_path": "/usr/sbin/cephadm", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephfs_mirror_action_update_interval": "2", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephfs_mirror_directory_scan_interval": "10", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephfs_mirror_mount_timeout": "10", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephfs_mirror_perf_stats_prio": "5", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephsqlite_blocklist_dead_locker": "true", 2026-02-25T08:40:43.886 INFO:teuthology.orchestra.run.trial006.stdout: "cephsqlite_lock_renewal_interval": "2000", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "cephx_cluster_require_signatures": "false", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "cephx_cluster_require_version": "2", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "cephx_require_signatures": "false", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "cephx_require_version": "2", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "cephx_service_require_signatures": "false", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "cephx_service_require_version": "2", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "cephx_sign_messages": "true", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "chdir": "", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "client_acl_type": "", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "client_asio_thread_count": "2", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "client_cache_mid": "0.750000", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "client_cache_size": "16384", 2026-02-25T08:40:43.887 INFO:teuthology.orchestra.run.trial006.stdout: "client_caps_release_delay": "5", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_check_pool_perm": "true", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_collect_and_send_global_metrics": "false", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_debug_force_sync_read": "false", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_debug_getattr_caps": "false", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_debug_inject_features": "", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_debug_inject_tick_delay": "0", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_die_on_failed_dentry_invalidate": "true", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_die_on_failed_remount": "false", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_dirsize_rbytes": "true", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_file_blockdiff_max_concurrent_object_scans": "16", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_force_lazyio": "false", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_fs": "", 2026-02-25T08:40:43.888 INFO:teuthology.orchestra.run.trial006.stdout: "client_fscrypt_as": "true", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_fscrypt_dummy_encryption": "false", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_inject_fixed_oldest_tid": "false", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_inject_release_failure": "false", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_inject_write_delay_secs": "0", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_max_inline_size": "4096", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_max_retries_on_remount_failure": "5", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_mds_namespace": "", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_metadata": "", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_mount_gid": "-1", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_mount_timeout": "300", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_mount_uid": "-1", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_mountpoint": "/", 2026-02-25T08:40:43.889 INFO:teuthology.orchestra.run.trial006.stdout: "client_notify_timeout": "10", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_oc": "true", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_oc_max_dirty": "104857600", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_oc_max_dirty_age": "5.000000", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_oc_max_objects": "1000", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_oc_size": "209715200", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_oc_target_dirty": "8388608", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_permissions": "true", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_quota": "true", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_quota_df": "true", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_readahead_max_bytes": "0", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_readahead_max_periods": "4", 2026-02-25T08:40:43.890 INFO:teuthology.orchestra.run.trial006.stdout: "client_readahead_min": "131072", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "client_reconnect_stale": "false", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "client_respect_subvolume_snapshot_visibility": "false", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "client_shutdown_timeout": "30", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "client_snapdir": ".snap", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "client_tick_interval": "1", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "client_trace": "", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "client_try_dentry_invalidate": "false", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "client_use_faked_inos": "false", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "client_use_random_mds": "false", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "clog_to_graylog": "false", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "clog_to_graylog_host": "127.0.0.1", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "clog_to_graylog_port": "12201", 2026-02-25T08:40:43.891 INFO:teuthology.orchestra.run.trial006.stdout: "clog_to_monitors": "default=true", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "clog_to_syslog": "false", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "clog_to_syslog_level": "info", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "cluster_addr": "-", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "cluster_network": "", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "cluster_network_interface": "", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "compressor_zlib_isal": "false", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "compressor_zlib_level": "5", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "compressor_zlib_winsize": "-15", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "compressor_zstd_level": "1", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "crash_dir": "/var/lib/ceph/crash", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_allow_pg_split": "true", 2026-02-25T08:40:43.892 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_bluestore_cpu_set": "", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_bluestore_num_threads": "6", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_cpu_num": "0", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_cpu_set": "", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_osd_obc_lru_size": "512", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_osd_scheduler_concurrency": "0", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_osd_stat_interval": "0", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_poll_mode": "false", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_reactor_idle_poll_time_us": "200", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_reactor_io_latency_goal_ms": "0.000000", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crimson_reactor_task_quota_ms": "0.500000", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crush_location": "", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crush_location_hook": "", 2026-02-25T08:40:43.893 INFO:teuthology.orchestra.run.trial006.stdout: "crush_location_hook_timeout": "10", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "d4n_writecache_enabled": "false", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "daemonize": "false", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "daos_pool": "tank", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "dbstore_db_name_prefix": "dbstore", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "debug_alienstore": "0/5", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "debug_allow_any_pool_priority": "false", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "debug_asok": "1/5", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "debug_asok_assert_abort": "false", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "debug_asserts_on_shutdown": "true", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "debug_auth": "20/20", 2026-02-25T08:40:43.894 INFO:teuthology.orchestra.run.trial006.stdout: "debug_bdev": "1/3", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_bluefs": "1/5", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_bluestore": "1/5", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_bluestore_compression": "1/5", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_buffer": "0/1", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_ceph_dedup": "0/5", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_ceph_exporter": "1/5", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_cephfs_mirror": "0/5", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_cephsqlite": "0/5", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_client": "0/5", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_compressor": "1/5", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_context": "0/1", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_crimson_interrupt": "0/5", 2026-02-25T08:40:43.895 INFO:teuthology.orchestra.run.trial006.stdout: "debug_crush": "1/1", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_crypto": "1/5", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_cyanstore": "0/5", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_deliberately_leak_memory": "false", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_disable_randomized_ping": "false", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_dpdk": "1/5", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_eventtrace": "1/5", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_filer": "0/1", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_filestore": "1/3", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_finisher": "1/1", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_fuse": "1/5", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_heartbeat_testing_span": "0", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_heartbeatmap": "1/5", 2026-02-25T08:40:43.896 INFO:teuthology.orchestra.run.trial006.stdout: "debug_immutable_obj_cache": "0/5", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_javaclient": "1/5", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_journal": "1/3", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_journaler": "0/5", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_kstore": "1/5", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_lockdep": "0/1", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_mclock": "1/5", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_mds": "1/5", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_mds_balancer": "1/5", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_mds_locker": "1/5", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_mds_log": "1/5", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_mds_log_expire": "1/5", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_mds_migrator": "1/5", 2026-02-25T08:40:43.897 INFO:teuthology.orchestra.run.trial006.stdout: "debug_mds_quiesce": "3/5", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_memstore": "1/5", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_mgr": "2/5", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_mgrc": "1/5", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_mon": "20/20", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_monc": "0/10", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_ms": "1/1", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_none": "0/5", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_objclass": "0/5", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_objectcacher": "0/5", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_objecter": "0/1", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_optracker": "0/5", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_osd": "1/5", 2026-02-25T08:40:43.898 INFO:teuthology.orchestra.run.trial006.stdout: "debug_paxos": "20/20", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_perfcounter": "1/5", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_prioritycache": "1/5", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rados": "0/5", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rbd": "0/5", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rbd_mirror": "0/5", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rbd_pwl": "0/5", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rbd_replay": "0/5", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_refs": "0/0", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_reserver": "1/1", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rgw": "1/5", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rgw_access": "1/5", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rgw_bucket_logging": "1/5", 2026-02-25T08:40:43.899 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rgw_datacache": "1/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rgw_dbstore": "1/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rgw_dedup": "1/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rgw_flight": "1/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rgw_lifecycle": "1/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rgw_notification": "1/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rgw_restore": "1/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rgw_sync": "1/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_rocksdb": "4/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore": "0/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_backref": "0/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_cache": "0/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_cleaner": "0/5", 2026-02-25T08:40:43.900 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_device": "0/5", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_epm": "0/5", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_fixedkv_tree": "0/5", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_journal": "0/5", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_lba": "0/5", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_odata": "0/5", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_omap": "0/5", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_onode": "0/5", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_t": "0/5", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_seastore_tm": "0/5", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_striper": "0/1", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_test": "0/5", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_throttle": "1/1", 2026-02-25T08:40:43.901 INFO:teuthology.orchestra.run.trial006.stdout: "debug_timer": "0/1", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "debug_tp": "0/5", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "debug_trace": "1/5", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "defer_client_eviction_on_laggy_osds": "false", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "device_failure_prediction_mode": "none", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "ec_extent_cache_size": "10485760", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "ec_pdw_write_mode": "0", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "enable_availability_tracking": "false", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "crimson", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "err_to_graylog": "false", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "err_to_journald": "false", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "err_to_stderr": "true", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "err_to_syslog": "false", 2026-02-25T08:40:43.902 INFO:teuthology.orchestra.run.trial006.stdout: "event_tracing": "false", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "exporter_addr": "0.0.0.0", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "exporter_cert_file": "", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "exporter_http_port": "9926", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "exporter_key_file": "", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "exporter_prio_limit": "5", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "exporter_sock_dir": "/var/run/ceph/", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "exporter_sort_metrics": "true", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "exporter_stats_period": "5", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "fake_statfs_for_testing": "0", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "fatal_signal_handlers": "true", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "filer_max_purge_ops": "10", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "filer_max_truncate_ops": "128", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_apply_finisher_threads": "1", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_blackhole": "false", 2026-02-25T08:40:43.903 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_btrfs_clone_range": "true", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_btrfs_snap": "true", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_caller_concurrency": "10", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_collect_device_partition_information": "true", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_commit_timeout": "600.000000", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_debug_inject_read_err": "false", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_debug_omap_check": "false", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_debug_verify_split": "false", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_dump_file": "", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_expected_throughput_ops": "200.000000", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_fadvise": "true", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_fail_eio": "true", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_fd_cache_shards": "16", 2026-02-25T08:40:43.904 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_fd_cache_size": "128", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_fiemap": "false", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_fiemap_threshold": "4096", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_fsync_flushes_journal_data": "false", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_index_retry_probability": "0.000000", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_inject_stall": "0", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_journal_parallel": "false", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_journal_trailing": "false", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_journal_writeahead": "false", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_kill_at": "0", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_alloc_hint_size": "1048576", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_inline_xattr_size": "0", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_inline_xattr_size_other": "512", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_inline_xattrs": "0", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2026-02-25T08:40:43.905 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_inline_xattrs_other": "2", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_inline_xattrs_xfs": "10", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_sync_interval": "5.000000", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_xattr_value_size": "0", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_xattr_value_size_other": "1024", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_merge_threshold": "-10", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_min_sync_interval": "0.010000", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_odsync_write": "false", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_omap_backend": "rocksdb", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_omap_backend_path": "", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_omap_header_cache_size": "1024", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_ondisk_finisher_threads": "1", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_op_thread_suicide_timeout": "180", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_op_thread_timeout": "60", 2026-02-25T08:40:43.906 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_op_threads": "2", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_punch_hole": "false", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_queue_high_threshhold": "0.900000", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_queue_low_threshhold": "0.300000", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_queue_max_bytes": "104857600", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_queue_max_ops": "50", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_seek_data_hole": "false", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_sloppy_crc": "false", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_sloppy_crc_block_size": "65536", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_splice": "false", 2026-02-25T08:40:43.907 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_split_multiple": "2", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_split_rand_factor": "20", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_update_to": "1000", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_enable": "true", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_xfs_extsize": "false", 2026-02-25T08:40:43.908 INFO:teuthology.orchestra.run.trial006.stdout: "filestore_zfs_snap": "false", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fio_dir": "/tmp/fio", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fsid": "b1b76914-ff72-4771-8729-7fc8b1bbfc83", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_allow_other": "true", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_atomic_o_trunc": "true", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_big_writes": "true", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_debug": "false", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_default_permissions": "false", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_disable_pagecache": "false", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_max_write": "0", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_multithreaded": "true", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_require_active_mds": "true", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_set_user_groups": "true", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_splice_move": "true", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_splice_read": "true", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_splice_write": "true", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_syncfs_on_mksnap": "true", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "fuse_use_invalidate_cb": "true", 2026-02-25T08:40:43.909 INFO:teuthology.orchestra.run.trial006.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "gss_target_name": "ceph", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "heartbeat_file": "", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "heartbeat_inject_failure": "0", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "heartbeat_interval": "5", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "host": "trial006", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_max_inflight_ops": "128", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_max_size": "1073741824", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_qos_bps_burst": "0", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_qos_bps_limit": "0", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_qos_iops_burst": "0", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_qos_iops_limit": "0", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "immutable_object_cache_watermark": "0.900000", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "inject_early_sigterm": "false", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "jaeger_agent_port": "6799", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "jaeger_tracing_enable": "false", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_aio": "true", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_align_min_size": "65536", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_block_align": "true", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_block_size": "4096", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_dio": "true", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_discard": "false", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_force_aio": "false", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_ignore_corruption": "false", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_max_write_bytes": "10485760", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_max_write_entries": "100", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_replay_from": "0", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_throttle_high_multiple": "0.000000", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_throttle_high_threshhold": "0.900000", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_throttle_low_threshhold": "0.600000", 2026-02-25T08:40:43.911 INFO:teuthology.orchestra.run.trial006.stdout: "journal_throttle_max_multiple": "0.000000", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "journal_write_header_frequency": "0", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "journal_zero_on_create": "false", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "journaler_prefetch_periods": "10", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "journaler_prezero_periods": "5", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "journaler_write_head_interval": "15", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "key": "", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "keyfile": "", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "librados_thread_count": "2", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "lockdep": "false", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "lockdep_force_backtrace": "false", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "log_coarse_timestamps": "true", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "log_flush_on_exit": "false", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "log_graylog_host": "127.0.0.1", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "log_graylog_port": "12201", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "log_max_new": "1000", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "log_max_recent": "10000", 2026-02-25T08:40:43.912 INFO:teuthology.orchestra.run.trial006.stdout: "log_stderr_prefix": "", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "log_stop_at_utilization": "0.970000", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "log_to_file": "true", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "log_to_graylog": "false", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "log_to_journald": "false", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "log_to_stderr": "false", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "log_to_syslog": "false", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "max_rotating_auth_attempts": "10", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_action_on_write_error": "1", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_allow_async_dirops": "true", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_allow_batched_ops": "true", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_alternate_name_max": "8192", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_asio_thread_count": "2", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_export_pin": "true", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_fragment_dirs": "true", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_fragment_interval": "5", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_fragment_size_max": "100000", 2026-02-25T08:40:43.913 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_idle_threshold": "0.000000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_interval": "10", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_max": "-1", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_max_until": "-1", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_merge_size": "50", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_midchunk": "0.300000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_min_rebalance": "0.100000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_min_start": "0.200000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_minchunk": "0.001000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_mode": "0", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_need_max": "1.200000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_need_min": "0.800000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_overload_epochs": "2", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_replicate_threshold": "8000.000000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_sample_interval": "3.000000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_split_bits": "3", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_split_rd": "25000.000000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_split_size": "10000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_split_wr": "10000.000000", 2026-02-25T08:40:43.914 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_target_decay": "10.000000", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_beacon_grace": "15.000000", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_beacon_interval": "4.000000", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_beacon_mon_down_grace": "60", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_memory_limit": "4294967296", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_mid": "0.700000", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_quiesce_delay": "0", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_quiesce_sleep": "200", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_quiesce_splitauth": "true", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_quiesce_threshold": "524288", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_release_free_interval": "10", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_reservation": "0.050000", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_trim_decay_rate": "1.000000", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_trim_interval": "1", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cache_trim_threshold": "262144", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_client_delegate_inos_pct": "50", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_client_prealloc_inos": "1000", 2026-02-25T08:40:43.915 INFO:teuthology.orchestra.run.trial006.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_connect_bootstrapping": "false", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_damage_table_max_entries": "10000", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_debug_auth_pins": "false", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_debug_frag": "false", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_debug_scatterstat": "false", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_debug_subtrees": "false", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_decay_halflife": "5.000000", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_default_dir_hash": "2", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_defer_session_stale": "true", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_delay_journal_replay_for_testing": "0", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_deny_all_reconnect": "false", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_dir_keys_per_op": "16384", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_dir_max_commit_size": "10", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_dir_max_entries": "0", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_dir_prefetch": "true", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_dirstat_min_interval": "1.000000", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_dump_cache_after_rejoin": "false", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_dump_cache_on_map": "false", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_dump_cache_threshold_file": "0", 2026-02-25T08:40:43.916 INFO:teuthology.orchestra.run.trial006.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_early_reply": "true", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_enable_op_tracker": "true", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_enforce_unique_name": "true", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_export_ephemeral_distributed": "true", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_export_ephemeral_random": "true", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_export_ephemeral_random_max": "0.010000", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_extraordinary_events_dump_interval": "0", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_file_blockdiff_max_concurrent_object_scans": "16", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_forward_all_requests_to_auth": "false", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_freeze_tree_timeout": "30.000000", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_fscrypt_last_block_max_size": "4096", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_go_bad_corrupt_dentry": "true", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_health_cache_threshold": "1.500000", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_health_summarize_threshold": "10", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_heartbeat_grace": "15.000000", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_heartbeat_reset_grace": "1000", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_inject_health_dummy": "false", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2026-02-25T08:40:43.917 INFO:teuthology.orchestra.run.trial006.stdout: "mds_inject_migrator_session_race": "false", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_inject_skip_replaying_inotable": "false", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_join_fs": "", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_journal_format": "1", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_after_journal_logs_flushed": "false", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_create_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_dirfrag_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_export_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_import_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_journal_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_journal_expire_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_journal_replay_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_link_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_mdstable_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_openc_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_rename_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_kill_shutdown_at": "0", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_event_large_threshold": "524288", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_events_per_segment": "1024", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_max_events": "-1", 2026-02-25T08:40:43.918 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_max_segments": "128", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_minor_segments_per_major_segment": "16", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_pause": "false", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_segment_size": "0", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_skip_corrupt_events": "false", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_skip_unbounded_events": "false", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_trim_decay_rate": "1.000000", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_trim_threshold": "128", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_trim_upkeep_interval": "1000", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_log_warn_factor": "2.000000", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_max_caps_per_client": "1048576", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_max_completed_flushes": "100000", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_max_completed_requests": "100000", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_max_export_size": "20971520", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_max_file_recover": "32", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_max_purge_files": "64", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_max_purge_ops": "8192", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_max_scrub_ops_in_progress": "5", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_max_snaps_per_dir": "100", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_metrics_update_interval": "2", 2026-02-25T08:40:43.919 INFO:teuthology.orchestra.run.trial006.stdout: "mds_min_caps_per_client": "100", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_min_caps_working_set": "10000", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_mon_shutdown_timeout": "5.000000", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_numa_node": "-1", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_oft_prefetch_dirfrags": "false", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_op_complaint_time": "30.000000", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_op_history_duration": "600", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_op_history_size": "20", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_op_history_slow_op_size": "20", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_op_history_slow_op_threshold": "10", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_op_log_threshold": "5", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_ping_grace": "15", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_ping_interval": "5", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_recall_global_max_decay_threshold": "131072", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_recall_max_caps": "30000", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_recall_max_decay_rate": "1.500000", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_recall_max_decay_threshold": "131072", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_recall_warning_decay_rate": "60.000000", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_recall_warning_threshold": "262144", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_reconnect_timeout": "45.000000", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_replay_interval": "1.000000", 2026-02-25T08:40:43.920 INFO:teuthology.orchestra.run.trial006.stdout: "mds_replay_unsafe_with_closed_session": "false", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_request_load_average_decay_rate": "60.000000", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_root_ino_gid": "0", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_root_ino_uid": "0", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_scatter_nudge_interval": "5.000000", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_scrub_stats_review_period": "1", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_server_dispatch_client_request_delay": "0", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_session_blocklist_on_evict": "true", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_session_blocklist_on_timeout": "true", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_session_cache_liveness_magnitude": "10", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_session_cap_acquisition_throttle": "100000", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_session_metadata_threshold": "16777216", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_sessionmap_keys_per_op": "1024", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_shutdown_check": "0", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_skip_ino": "0", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_sleep_rank_change": "0.000000", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_snap_max_uid": "4294967294", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_snap_min_uid": "0", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_snap_rstat": "false", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_standby_replay_damaged": "false", 2026-02-25T08:40:43.921 INFO:teuthology.orchestra.run.trial006.stdout: "mds_symlink_recovery": "true", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mds_task_status_update_interval": "2.000000", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mds_thrash_exports": "0", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mds_thrash_fragments": "0", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mds_tick_interval": "5.000000", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mds_use_global_snaprealm_seq_for_subvol": "true", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mds_valgrind_exit": "false", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mds_verify_backtrace": "1", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mds_verify_scatter": "false", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mds_wipe_ino_prealloc": "false", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mds_wipe_sessions": "false", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mempool_debug": "false", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "memstore_debug_omit_block_device_write": "false", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "memstore_device_bytes": "1073741824", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "memstore_page_set": "false", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "memstore_page_size": "65536", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_client_bytes": "134217728", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_client_messages": "512", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_connect_retry_interval": "1.000000", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_disabled_modules": "", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_enable_op_tracker": "true", 2026-02-25T08:40:43.922 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_initial_modules": "iostat nfs", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_max_pg_creating": "1024", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_max_pg_num_change": "128", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_mds_bytes": "134217728", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_mds_messages": "128", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_mon_bytes": "134217728", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_mon_messages": "128", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_num_op_tracker_shard": "32", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_op_complaint_time": "30.000000", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_op_history_duration": "600", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_op_history_size": "20", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_op_history_slow_op_size": "20", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_op_log_threshold": "5", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_osd_bytes": "536870912", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_osd_messages": "8192", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_pool": "true", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_service_beacon_grace": "60.000000", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_standby_modules": "true", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_stats_period": "5", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_stats_threshold": "5", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_tick_period": "2", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mgr_ttl_cache_expire_seconds": "0", 2026-02-25T08:40:43.923 INFO:teuthology.orchestra.run.trial006.stdout: "mon_accept_timeout_factor": "2.000000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_allow_pool_delete": "true", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_allow_pool_size_one": "true", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_auth_validate_all_caps": "true", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_bytes": "104857600", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_directed_command_retry": "2", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_hunt_interval": "3.000000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_hunt_on_resend": "true", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_hunt_parallel": "3", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_log_interval": "1.000000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_max_log_entries_per_message": "1000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_ping_interval": "10.000000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_ping_timeout": "30.000000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_client_target_rank": "-1", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_clock_drift_allowed": "1.000000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cluster_log_level": "debug", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cluster_log_to_file": "true", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cluster_log_to_graylog": "false", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cluster_log_to_graylog_port": "12201", 2026-02-25T08:40:43.924 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cluster_log_to_journald": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cluster_log_to_stderr": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cluster_log_to_syslog": "default=false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_compact_on_bootstrap": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_compact_on_start": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_compact_on_trim": "true", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_con_tracker_persist_interval": "10", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_con_tracker_score_halflife": "43200", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_config_key_max_entry_size": "65536", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_cpu_threads": "4", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_crush_min_required_version": "hammer", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_daemon_bytes": "419430400", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_data_avail_crit": "5", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_data_avail_warn": "5", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_data_size_warn": "16106127360", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_debug_block_osdmap_trim": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_debug_deprecated_as_obsolete": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_debug_dump_json": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_debug_dump_transactions": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_debug_extra_checks": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_debug_no_initial_persistent_features": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_debug_no_require_squid": "false", 2026-02-25T08:40:43.925 INFO:teuthology.orchestra.run.trial006.stdout: "mon_debug_no_require_tentacle": "false", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_delta_reset_interval": "10.000000", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_dns_srv_name": "ceph-mon", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_down_added_grace": "180", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_down_mkfs_grace": "120", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_down_uptime_grace": "60", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_election_default_strategy": "1", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_election_timeout": "5.000000", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_elector_ping_divisor": "2", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_elector_ping_timeout": "2.000000", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_enable_op_tracker": "true", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_fake_pool_delete": "false", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_force_quorum_join": "false", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_fsmap_prune_threshold": "300", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_globalid_prealloc": "10000", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_health_detail_to_clog": "false", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_health_log_update_period": "5", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_health_max_detail": "50", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_health_to_clog": "true", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_health_to_clog_interval": "600", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_host": "10.20.193.6", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_host_override": "", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_initial_members": "", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2026-02-25T08:40:43.926 INFO:teuthology.orchestra.run.trial006.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_inject_transaction_delay_max": "10.000000", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_keyvaluedb": "rocksdb", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_lease": "5.000000", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_lease_renew_interval_factor": "0.600000", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_log_full_interval": "50", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_log_max": "10000", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_log_max_summary": "50", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_max_log_entries_per_event": "4096", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_max_log_epochs": "500", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_max_mdsmap_epochs": "500", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_max_mgrmap_epochs": "500", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_max_nvmeof_epochs": "500", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_max_osd": "10000", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_max_pg_per_osd": "10000", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_max_pool_pg_num": "65536", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_max_snap_prune_per_epoch": "100", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_mds_blocklist_interval": "86400.000000", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_mds_force_trim_to": "0", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_mds_skip_sanity": "false", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_memory_autotune": "true", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_memory_target": "2147483648", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_mgr_beacon_grace": "30", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_mgr_digest_period": "5", 2026-02-25T08:40:43.927 INFO:teuthology.orchestra.run.trial006.stdout: "mon_mgr_inactive_grace": "60", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_mgr_mkfs_grace": "240", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_min_osdmap_epochs": "50", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_netsplit_grace_period": "9", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_nvmeofgw_beacon_grace": "7", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_nvmeofgw_beacons_till_ack": "15", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_nvmeofgw_delete_grace": "900", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_nvmeofgw_skip_failovers_interval": "16", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_nvmeofgw_wrong_map_ignore_sec": "15", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_op_complaint_time": "30", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_op_history_duration": "600", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_op_history_size": "20", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_op_history_slow_op_size": "20", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_op_history_slow_op_threshold": "10", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_op_log_threshold": "5", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_adjust_down_out_interval": "true", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_auto_mark_in": "false", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_auto_mark_new_in": "true", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_cache_size": "500", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_cache_size_min": "134217728", 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_crush_smoke_test": "true", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_destroyed_out_interval": "600", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_down_out_interval": "600", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_down_out_subtree_limit": "rack", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_force_trim_to": "0", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_full_ratio": "0.950000", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_laggy_halflife": "3600", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_laggy_max_interval": "300", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_laggy_weight": "0.300000", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_max_initial_pgs": "1024", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_min_down_reporters": "2", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_min_in_ratio": "0.750000", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_min_up_ratio": "0.300000", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_nearfull_ratio": "0.850000", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_prime_pg_temp": "true", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_report_timeout": "900", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_reporter_subtree_level": "osd", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_warn_num_repaired": "10", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osd_warn_op_age": "32.000000", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osdmap_full_prune_enabled": "true", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osdmap_full_prune_interval": "2", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osdmap_full_prune_min": "15", 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial006.stdout: "mon_osdmap_full_prune_txsize": "2", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_pg_stuck_threshold": "60", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_pg_warn_min_objects": "10000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_pg_warn_min_per_osd": "0", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_pg_warn_min_pool_objects": "1000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_pool_quota_crit_threshold": "0", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_pool_quota_warn_threshold": "0", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_probe_timeout": "2.000000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_reweight_max_change": "0.050000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_reweight_max_osds": "4", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_reweight_min_bytes_per_osd": "10", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_reweight_min_pgs_per_osd": "4", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_scrub_interval": "86400", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_scrub_max_keys": "100", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_scrub_timeout": "300", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_session_timeout": "300", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_smart_report_timeout": "5", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_stat_smooth_intervals": "6", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_stretch_pool_min_size": "2", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_stretch_pool_size": "4", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_subscribe_interval": "86400.000000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_sync_debug": "false", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_sync_max_payload_keys": "2000", 2026-02-25T08:40:43.930 INFO:teuthology.orchestra.run.trial006.stdout: "mon_sync_max_payload_size": "1048576", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_sync_provider_kill_at": "0", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_sync_requester_kill_at": "0", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_sync_timeout": "60.000000", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_target_pg_per_osd": "100", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_tick_interval": "5", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_timecheck_interval": "300.000000", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_timecheck_skew_interval": "30.000000", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_use_min_delay_socket": "false", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_older_version_delay": "604800", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_filestore_osds": "false", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_misplaced": "false", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_older_version": "true", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_pool_no_app": "true", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_pool_no_app_grace": "300", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_pool_no_redundancy": "false", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_on_too_few_osds": "false", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2026-02-25T08:40:43.931 INFO:teuthology.orchestra.run.trial006.stdout: "monmap": "", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "motr_tracing_enabled": "false", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_op_threads": "3", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_buffer_size": "131072", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_cm": "false", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_device_name": "", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_dscp": "96", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_enable_hugepage": "false", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_gid_idx": "0", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_local_gid": "", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_polling_us": "1000", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_port_num": "1", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_receive_buffers": "32768", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_receive_queue_len": "4096", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_roce_ver": "1", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_send_buffers": "1024", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_sl": "3", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_support_srq": "true", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_rdma_type": "ib", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_async_reap_threshold": "5", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_bind_before_connect": "false", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_bind_ipv4": "true", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_bind_ipv6": "false", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_bind_msgr1": "true", 2026-02-25T08:40:43.932 INFO:teuthology.orchestra.run.trial006.stdout: "ms_bind_msgr2": "true", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_bind_port_max": "7568", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_bind_port_min": "6800", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_bind_prefer_ipv4": "false", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_bind_retry_count": "3", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_bind_retry_delay": "5", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_blackhole_client": "false", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_blackhole_mds": "false", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_blackhole_mgr": "false", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_blackhole_mon": "false", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_blackhole_osd": "false", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_client_mode": "crc", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_client_throttle_retry_time_interval": "5000", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_cluster_mode": "crc", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_cluster_type": "", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_compress_secure": "false", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_connection_idle_timeout": "900", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_connection_ready_timeout": "10", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_crc_data": "true", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_crc_header": "true", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_die_on_bad_msg": "false", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_die_on_bug": "true", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_die_on_old_message": "true", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_die_on_skipped_message": "false", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_die_on_unhandled_msg": "false", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dispatch_throttle_bytes": "104857600", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_coremask": "0xF", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_debug_allow_loopback": "false", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_devs_allowlist": "", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_enable_tso": "true", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2026-02-25T08:40:43.933 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_host_ipv4_addr": "", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_hugepages": "", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_hw_flow_control": "true", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_lro": "true", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_memory_channel": "4", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_pmd": "", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_port_id": "0", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dump_corrupt_message_level": "1", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_dump_on_send": "false", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_initial_backoff": "0.200000", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_inject_delay_max": "1.000000", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_inject_delay_probability": "0.000000", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_inject_delay_type": "", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_inject_internal_delays": "0.000000", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_inject_network_congestion": "0", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_inject_socket_failures": "0", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_learn_addr_from_peer": "true", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_max_accept_failures": "4", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_max_backoff": "15.000000", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_mon_client_mode": "crc", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_mon_cluster_mode": "crc", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_mon_service_mode": "crc", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_osd_compress_min_size": "1024", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_osd_compress_mode": "none", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_osd_compression_algorithm": "snappy", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_pq_min_cost": "65536", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_public_type": "", 2026-02-25T08:40:43.934 INFO:teuthology.orchestra.run.trial006.stdout: "ms_service_mode": "crc", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "ms_tcp_listen_backlog": "512", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "ms_tcp_nodelay": "true", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "ms_tcp_prefetch_max_size": "65536", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "ms_tcp_rcvbuf": "0", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "ms_time_events_min_wait_interval": "1000", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "ms_type": "async+posix", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "no_config_file": "false", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "nvmeof_mon_client_connect_panic": "30", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "nvmeof_mon_client_tick_period": "1", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "objecter_completion_locks_per_session": "32", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "objecter_debug_inject_relock_delay": "false", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "objecter_inflight_op_bytes": "104857600", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "objecter_inflight_ops": "1024", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "objecter_inject_no_watch_ping": "false", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "objecter_retry_writes_after_first_reply": "false", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "objecter_tick_interval": "5.000000", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "objecter_timeout": "10.000000", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "objectstore_blackhole": "false", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "openssl_conf": "", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "osd_agent_delay_time": "5.000000", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "osd_agent_hist_halflife": "1000", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "osd_agent_max_low_ops": "2", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "osd_agent_max_ops": "4", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "osd_agent_min_evict_effort": "0.100000", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "osd_agent_quantize_effort": "0.100000", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "osd_agent_slop": "0.020000", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "osd_aggregated_slow_ops_logging": "true", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "osd_allow_recovery_below_min_size": "true", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "osd_asio_thread_count": "2", 2026-02-25T08:40:43.935 INFO:teuthology.orchestra.run.trial006.stdout: "osd_async_recovery_min_cost": "100", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_auto_mark_unfound_lost": "false", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_backfill_retry_interval": "30.000000", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_backfill_scan_max": "512", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_backfill_scan_min": "64", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_backoff_on_degraded": "false", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_backoff_on_peering": "false", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_backoff_on_unfound": "true", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_beacon_report_interval": "300", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_bench_duration": "30", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_bench_large_size_max_throughput": "104857600", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_bench_max_block_size": "67108864", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_bench_small_size_max_iops": "100", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_blkin_trace_all": "false", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_blocked_scrub_grace_period": "120", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_check_for_log_corruption": "false", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_check_max_object_name_len_on_startup": "true", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo sem_set", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo sem_set", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_class_update_on_start": "true", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_client_message_cap": "256", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_client_message_size_cap": "524288000", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_client_op_priority": "63", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_client_watch_timeout": "30", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_command_max_records": "256", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_compact_on_start": "false", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_copyfrom_max_chunk": "8388608", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_crush_chooseleaf_type": "0", 2026-02-25T08:40:43.936 INFO:teuthology.orchestra.run.trial006.stdout: "osd_crush_initial_weight": "-1.000000", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_crush_update_on_start": "true", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_crush_update_weight_set": "true", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_drop_ping_duration": "0", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_drop_ping_probability": "0.000000", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_feed_pullee": "-1", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_inject_copyfrom_error": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_misdirected_ops": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_no_acting_change": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_no_purge_strays": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_op_order": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_pg_log_writeout": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_pretend_recovery_active": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_random_push_read_error": "0.000000", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_shutdown": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_trim_objects": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_verify_cached_snaps": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_verify_missing_on_start": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_verify_snaps": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_debug_verify_stray_on_activate": "false", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_deep_scrub_interval": "604800.000000", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_deep_scrub_interval_cv": "0.200000", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_deep_scrub_keys": "1024", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2026-02-25T08:40:43.937 INFO:teuthology.orchestra.run.trial006.stdout: "osd_deep_scrub_stride": "4194304", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_default_data_pool_replay_window": "5", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_default_notify_timeout": "30", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_delete_sleep": "0.000000", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_delete_sleep_hdd": "5.000000", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_delete_sleep_hybrid": "1.000000", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_delete_sleep_ssd": "1.000000", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_discard_disconnected_ops": "true", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_ec_partial_reads": "true", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_enable_op_tracker": "true", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_extblkdev_plugins": "vdo", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_failsafe_full_ratio": "0.970000", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_fast_fail_on_connection_refused": "true", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_fast_info": "true", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_fast_shutdown": "true", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_fast_shutdown_notify_mon": "true", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_fast_shutdown_timeout": "15", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_find_best_info_ignore_history_les": "false", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_force_auth_primary_missing_objects": "100", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_function_tracing": "false", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_heartbeat_grace": "20", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_heartbeat_interval": "6", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_heartbeat_min_peers": "10", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_heartbeat_min_size": "2000", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_heartbeat_stale": "600", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_hit_set_max_size": "100000", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_hit_set_min_size": "1000", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_hit_set_namespace": ".ceph-internal", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_ignore_stale_divergent_priors": "false", 2026-02-25T08:40:43.938 INFO:teuthology.orchestra.run.trial006.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_inject_failure_on_pg_removal": "false", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_journal_flush_on_shutdown": "true", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_journal_size": "5120", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_kill_backfill_at": "0", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_loop_before_reset_tphandle": "64", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_map_cache_size": "50", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_map_dedup": "true", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_map_message_max": "40", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_map_message_max_bytes": "10485760", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_map_share_max_epochs": "40", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_attr_name_len": "100", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_attr_size": "0", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_backfills": "1", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_markdown_count": "5", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_markdown_period": "600", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_object_name_len": "2048", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_object_namespace_len": "256", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_object_size": "134217728", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_omap_entries_per_request": "1024", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_pg_blocked_by": "16", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_pg_log_entries": "10000", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_pgls": "1024", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_push_cost": "8388608", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_push_objects": "10", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_scrubs": "3", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_trimming_pgs": "2", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_write_op_reply_len": "64", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_max_write_size": "90", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_iops_capacity_low_threshold_hdd": "50.000000", 2026-02-25T08:40:43.939 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_iops_capacity_low_threshold_ssd": "1000.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_override_recovery_settings": "false", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_profile": "balanced", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_scheduler_client_wgt": "1", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mclock_skip_benchmark": "false", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_memory_base": "805306368", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_memory_cache_min": "134217728", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_memory_cache_resize_interval": "1.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_memory_expected_fragmentation": "0.150000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_memory_target": "4294967296", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_memory_target_autotune": "false", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_min_pg_log_entries": "250", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_min_recovery_priority": "0", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mon_heartbeat_interval": "30", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mon_report_interval": "5", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mon_report_max_in_flight": "2", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_mon_shutdown_timeout": "5.000000", 2026-02-25T08:40:43.940 INFO:teuthology.orchestra.run.trial006.stdout: "osd_num_cache_shards": "32", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_num_op_tracker_shard": "32", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_numa_auto_affinity": "true", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_numa_node": "-1", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_numa_prefer_iface": "true", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_object_clean_region_max_num_intervals": "10", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_objecter_finishers": "1", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_objectstore": "bluestore", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_objectstore_fuse": "false", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_objectstore_ideal_list_max": "64", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_objectstore_tracing": "false", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_complaint_time": "30.000000", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_history_duration": "600", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_history_size": "20", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_history_slow_op_size": "20", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_log_threshold": "5", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_num_shards": "0", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_num_shards_hdd": "1", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_num_shards_ssd": "8", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_num_threads_per_shard": "0", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_num_threads_per_shard_hdd": "5", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_pq_min_cost": "65536", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_queue": "mclock_scheduler", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_queue_cut_off": "high", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_thread_suicide_timeout": "150", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_op_thread_timeout": "15", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_open_classes_on_start": "true", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_os_flags": "0", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_peering_op_priority": "255", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pg_delete_cost": "1048576", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pg_delete_priority": "5", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2026-02-25T08:40:43.941 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pg_log_dups_tracked": "3000", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pg_log_trim_max": "10000", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pg_log_trim_min": "100", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pg_object_context_cache_count": "64", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_cache_min_evict_age": "0", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_cache_min_flush_age": "0", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_crimson": "true", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_crush_rule": "-1", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_ec_fast_read": "false", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_erasure_code_profile": "plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_flag_bulk": "false", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_flag_ec_optimizations": "false", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_flag_hashpspool": "true", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_flag_nodelete": "false", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_flag_nopgchange": "false", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_flag_nosizechange": "false", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_flags": "0", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_min_size": "2", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_pg_num": "32", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_pgp_num": "0", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_read_ratio": "70", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_size": "3", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_default_type": "replicated", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_erasure_code_stripe_unit": "0", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_pool_use_gmt_hitset": "true", 2026-02-25T08:40:43.942 INFO:teuthology.orchestra.run.trial006.stdout: "osd_push_per_object_cost": "1000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_read_ec_check_for_errors": "false", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recover_clone_overlap": "true", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recover_clone_overlap_limit": "10", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_cost": "20971520", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_delay_start": "0.000000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_max_active": "0", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_max_active_hdd": "3", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_max_active_ssd": "10", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_max_chunk": "8388608", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_max_single_start": "1", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_op_priority": "3", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_op_warn_multiple": "16", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_priority": "5", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_retry_interval": "30.000000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_sleep": "0.000000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_sleep_degraded": "0.000000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_sleep_degraded_hdd": "0.100000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_sleep_degraded_hybrid": "0.025000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_sleep_degraded_ssd": "0.000000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_sleep_hdd": "0.100000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_recovery_sleep_ssd": "0.000000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_requested_scrub_priority": "5", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_rollback_to_cluster_snap": "", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_auto_repair": "false", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_auto_repair_num_errors": "5", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_backoff_ratio": "0.660000", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_begin_hour": "0", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_begin_week_day": "0", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_chunk_max": "15", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_chunk_min": "5", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_cost": "52428800", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_disable_reservation_queuing": "false", 2026-02-25T08:40:43.943 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_during_recovery": "false", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_end_hour": "0", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_end_week_day": "0", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_event_cost": "4096", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_extended_sleep": "0.000000", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_invalid_stats": "true", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_load_threshold": "10.000000", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_max_interval": "604800.000000", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_max_preemptions": "5", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_min_interval": "86400.000000", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_priority": "5", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_retry_after_noscrub": "60", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_retry_delay": "30", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_retry_new_interval": "10", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_retry_pg_state": "60", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_retry_trimming": "10", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_scrub_sleep": "0.000000", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_shallow_scrub_chunk_max": "100", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_shallow_scrub_chunk_min": "50", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_shutdown_pgref_assert": "false", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_skip_check_past_interval_bounds": "false", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_skip_data_digest": "false", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_smart_report_timeout": "5", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_snap_trim_cost": "1048576", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_snap_trim_priority": "5", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_snap_trim_sleep": "0.000000", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_stats_update_period_not_scrubbing": "120", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_stats_update_period_scrubbing": "15", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_target_transaction_size": "30", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_tier_default_cache_hit_set_count": "4", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2026-02-25T08:40:43.944 INFO:teuthology.orchestra.run.trial006.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "osd_tier_default_cache_mode": "writeback", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "osd_tier_promote_max_objects_sec": "25", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "osd_tracing": "false", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "osd_use_stale_snap": "false", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "osdc_blkin_trace_all": "false", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "paxos_kill_at": "0", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "paxos_max_join_drift": "10", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "paxos_min": "500", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "paxos_min_wait": "0.050000", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "paxos_propose_interval": "1.000000", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "paxos_service_trim_max": "500", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "paxos_service_trim_max_multiplier": "20", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "paxos_service_trim_min": "10", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "paxos_stash_full_interval": "25", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "paxos_trim_max": "500", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "paxos_trim_min": "250", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "perf": "true", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "plugin_crypto_accelerator": "crypto_isal", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "plugin_dir": "/usr/lib64/ceph", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "pool_availability_update_interval": "1.000000", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "public_addr": "-", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "public_addrv": "", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "public_bind_addr": "-", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "public_network": "", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "public_network_interface": "", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "qat_compressor_busy_polling": "false", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "qat_compressor_enabled": "false", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "qat_compressor_session_max_number": "256", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "rados_mon_op_timeout": "0", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "rados_osd_op_timeout": "0", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "rados_replica_read_policy": "default", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "rados_replica_read_policy_on_objclass": "false", 2026-02-25T08:40:43.945 INFO:teuthology.orchestra.run.trial006.stdout: "rados_tracing": "false", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_atime_update_interval": "60", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_balance_parent_reads": "false", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_balance_snap_reads": "false", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_blkin_trace_all": "false", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_blocklist_expire_seconds": "0", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_blocklist_on_break_lock": "true", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_cache": "true", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_cache_block_writes_upfront": "false", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_cache_max_dirty": "25165824", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_cache_max_dirty_age": "1.000000", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_cache_max_dirty_object": "0", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_cache_policy": "writearound", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_cache_size": "33554432", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_cache_target_dirty": "16777216", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_cache_writethrough_until_flush": "true", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_clone_copy_on_read": "false", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_compression_hint": "none", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_concurrent_management_ops": "10", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_config_pool_override_update_timestamp": "0", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_default_clone_format": "auto", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_default_data_pool": "", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_default_features": "61", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_default_format": "2", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_default_map_options": "", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_default_order": "22", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_default_pool": "rbd", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_default_stripe_count": "0", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_default_stripe_unit": "0", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_disable_zero_copy_writes": "true", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_discard_granularity_bytes": "65536", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_discard_on_zeroed_write_same": "true", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_enable_alloc_hint": "true", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_io_scheduler": "simple", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_journal_commit_age": "5.000000", 2026-02-25T08:40:43.946 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_journal_max_payload_bytes": "16384", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_journal_object_flush_age": "0.000000", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_journal_object_flush_bytes": "1048576", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_journal_object_flush_interval": "0", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_journal_order": "24", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_journal_pool": "", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_journal_splay_width": "4", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_localize_parent_reads": "false", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_localize_snap_reads": "false", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_image_policy_type": "simple", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_image_state_check_interval": "30", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_memory_autotune": "true", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_memory_base": "805306368", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_memory_cache_min": "134217728", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_memory_target": "4294967296", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_perf_stats_prio": "5", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirroring_delete_delay": "0", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirroring_replay_delay": "0", 2026-02-25T08:40:43.947 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_move_to_trash_on_remove": "false", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_mtime_update_interval": "60", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_non_blocking_aio": "true", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_op_thread_timeout": "60", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_op_threads": "1", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_parent_cache_enabled": "false", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_persistent_cache_mode": "disabled", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_persistent_cache_path": "/tmp", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_persistent_cache_size": "1073741824", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_plugins": "", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_bps_burst": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_bps_burst_seconds": "1", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_bps_limit": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_exclude_ops": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_iops_burst": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_iops_burst_seconds": "1", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_iops_limit": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_read_bps_burst": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_read_bps_limit": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_read_iops_burst": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_read_iops_limit": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_schedule_tick_min": "50", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_write_bps_burst": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_write_bps_limit": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_write_iops_burst": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_qos_write_iops_limit": "0", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_quiesce_notification_attempts": "10", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_read_from_replica_policy": "default", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_readahead_max_bytes": "524288", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_readahead_trigger_requests": "10", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_request_timed_out_seconds": "30", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_skip_partial_discard": "true", 2026-02-25T08:40:43.948 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_tracing": "false", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_validate_names": "true", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rbd_validate_pool": "true", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "restapi_base_url": "", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "restapi_log_level": "", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_account_default_quota_max_objects": "-1", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_account_default_quota_max_size": "-1", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_acl_grants_max_num": "100", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_admin_entry": "admin", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_asio_assert_yielding": "false", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_backend_store": "rados", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_barbican_url": "", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_beast_enable_async": "true", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_counters_cache": "false", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_counters_cache_size": "10000", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_default_quota_max_size": "-1", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_eexist_override": "false", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_index_max_aio": "128", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_logging_obj_roll_time": "300", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_persistent_notif_num_shards": "11", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_quota_cache_size": "10000", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_quota_ttl": "600", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_bucket_sync_spawn_window": "20", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_cache_enabled": "true", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_cache_expiry_interval": "900", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_cache_lru_size": "25000", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_config_store": "rados", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_content_length_compat": "false", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_copy_obj_progress": "true", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_cors_rules_max_num": "100", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_cross_domain_policy": "", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_default_encryption_key": "", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_kmip_addr": "", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_kmip_ca_path": "", 2026-02-25T08:40:43.949 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_kmip_client_cert": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_kmip_client_key": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_kmip_kms_key_template": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_kmip_password": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_kmip_username": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_require_ssl": "true", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_backend": "vault", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_suppress_logs": "true", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_vault_addr": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_vault_auth": "token", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_vault_namespace": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_vault_prefix": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_vault_secret_engine": "transit", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_vault_ssl_cacert": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_vault_token_file": "", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_crypt_vault_verify_ssl": "true", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_curl_buffersize": "524288", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_curl_low_speed_limit": "1024", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_curl_low_speed_time": "300", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_curl_tcp_keepalive": "0", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_curl_wait_timeout_ms": "1000", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d3n_l1_fadvise": "4", 2026-02-25T08:40:43.950 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d3n_libaio_aio_num": "64", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d3n_libaio_aio_threads": "20", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_backend_address": "127.0.0.1:6379", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_cache_cleaning_interval": "1000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_l1_datacache_disk_reserve": "1073741824", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_l1_fadvise": "4", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_l1_write_open_flags": "4096", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_libaio_aio_num": "64", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_libaio_aio_threads": "20", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_d4n_local_rgw_address": "127.0.0.1:8000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_data_log_changes_size": "1000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_data_log_num_shards": "128", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_data_log_window": "30", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_data_notify_interval_msec": "0", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_data_sync_poll_interval": "20", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_data_sync_spawn_window": "20", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_debug_inject_set_olh_err": "0", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_default_data_log_backing": "fifo", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_default_realm_info_oid": "default.realm", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_default_region_info_oid": "default.region", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_default_zone_info_oid": "default.zone", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_defer_to_bucket_acls": "", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_delete_multi_obj_max_num": "1000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_disable_s3select": "false", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_admin_lim": "0.000000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_admin_res": "100.000000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_auth_lim": "0.000000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_auth_res": "200.000000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_data_lim": "0.000000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_data_res": "500.000000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_data_wgt": "500.000000", 2026-02-25T08:40:43.951 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_metadata_res": "500.000000", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dns_name": "", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dns_s3website_name": "", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dynamic_resharding": "true", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_enable_gc_threads": "true", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_enable_jwks_url_verification": "false", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_enable_lc_threads": "true", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_enable_mdsearch": "true", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_enable_ops_log": "false", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_enable_quota_threads": "true", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_enable_restore_threads": "true", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_enable_static_website": "false", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_enable_usage_log": "false", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_enforce_swift_acls": "true", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_exit_timeout_secs": "120", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_expose_bucket": "false", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_extended_http_attrs": "", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_filter": "none", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_frontends": "beast port=7480", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_gc_max_concurrent_io": "10", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_gc_max_deferred": "50", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_gc_max_objs": "32", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_gc_max_queue_size": "134213632", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_gc_max_trim_chunk": "16", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_gc_obj_min_wait": "7200", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_gc_processor_max_time": "3600", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_gc_processor_period": "3600", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_get_obj_max_req_size": "4194304", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_get_obj_window_size": "16777216", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_graceful_stop": "false", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_healthcheck_disabling_path": "", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_http_notif_connection_timeout": "5", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_http_notif_max_inflight": "8192", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_http_notif_message_timeout": "10", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ignore_get_invalid_range": "false", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_init_timeout": "300", 2026-02-25T08:40:43.952 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_kafka_connection_idle": "300", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_kafka_message_timeout": "5000", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_kafka_sleep_timeout": "10", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_accepted_admin_roles": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_accepted_reader_roles": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_admin_domain": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_admin_password": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_admin_password_path": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_admin_project": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_admin_tenant": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_admin_user": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_barbican_domain": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_barbican_password": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_barbican_project": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_barbican_tenant": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_barbican_user": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_implicit_tenants": "false", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_service_token_enabled": "false", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_token_cache_size": "10000", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_token_cache_ttl": "300", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_url": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_keystone_verify_ssl": "true", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lc_debug_interval": "-1", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lc_list_cnt": "1000", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lc_lock_max_time": "90", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lc_max_objs": "32", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lc_max_rules": "1000", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lc_max_worker": "3", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lc_max_wp_worker": "128", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lc_ordered_list_threshold": "500", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lc_thread_delay": "0", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ldap_dnattr": "uid", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ldap_searchfilter": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ldap_uri": "", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lfuda_sync_frequency": "60", 2026-02-25T08:40:43.953 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_list_bucket_min_readahead": "1000", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_list_buckets_max_chunk": "1000", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_log_http_headers": "", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_log_nonexistent_bucket": "false", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_log_object_name_utc": "false", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lua_enable": "true", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lua_max_memory_per_state": "128000", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_lua_max_runtime_per_state": "1000", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_attr_name_len": "0", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_attr_size": "0", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_attrs_num_in_req": "0", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_chunk_size": "4194304", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_concurrent_requests": "1024", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_control_aio": "8", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_dynamic_shards": "1999", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_listing_results": "5000", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_notify_retries": "10", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_objs_per_shard": "100000", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_put_param_size": "1048576", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_put_size": "5368709120", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_max_slo_entries": "1000", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_md_log_max_shards": "64", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_md_notify_interval_msec": "200", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_meta_sync_poll_interval": "20", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_meta_sync_spawn_window": "20", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_mime_types_file": "/etc/mime.types", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_mp_lock_max_time": "600", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_multi_obj_del_max_aio": "16", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_multipart_min_part_size": "5242880", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_multipart_part_upload_limit": "10000", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_fhcache_partitions": "3", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_fhcache_size": "2017", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_frontends": "rgw-nfs", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_lru_lanes": "5", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_max_gc": "300", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_namespace_expire_secs": "300", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_run_gc_threads": "false", 2026-02-25T08:40:43.954 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_run_lc_threads": "false", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_run_quota_threads": "false", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_run_restore_threads": "false", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_run_sync_thread": "false", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_s3_fast_attrs": "false", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_nfs_write_completion_interval_s": "10", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_num_async_rados_threads": "32", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_num_control_oids": "8", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_numa_node": "-1", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_obj_stripe_size": "4194304", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_obj_tombstone_cache_size": "1000", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_objexp_chunk_size": "100", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_objexp_gc_interval": "600", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_objexp_hints_num_shards": "127", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_olh_pending_timeout_sec": "3600", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_op_thread_suicide_timeout": "0", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_op_thread_timeout": "600", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_op_tracing": "false", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_opa_token": "", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_opa_url": "", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_opa_verify_ssl": "true", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ops_log_data_backlog": "5242880", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ops_log_rados": "false", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ops_log_socket_path": "", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_override_bucket_index_max_shards": "0", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_parquet_buffer_size": "16777216", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_period_push_interval": "2.000000", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_period_push_interval_max": "30.000000", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_period_root_pool": ".rgw.root", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_policy_reject_invalid_principals": "true", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_posix_cache_lanes": "3", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_posix_cache_lmdb_count": "3", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_posix_cache_max_buckets": "100", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_posix_cache_partitions": "3", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_posix_userdb_dir": "/var/lib/ceph/radosgw", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_print_continue": "true", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_print_prohibited_content_length": "false", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_put_obj_max_window_size": "67108864", 2026-02-25T08:40:43.955 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_put_obj_min_window_size": "16777216", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_rados_pool_recovery_priority": "5", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_rados_tracing": "false", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_ratelimit_interval": "60", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_realm": "", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_realm_id": "", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_realm_root_pool": ".rgw.root", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_redis_connection_pool_size": "512", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_region": "", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_region_root_pool": ".rgw.root", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_relaxed_region_enforcement": "false", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_relaxed_s3_bucket_names": "false", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_request_uri": "", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_reshard_batch_size": "64", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_reshard_bucket_lock_duration": "360", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_reshard_debug_interval": "-1", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_reshard_max_aio": "128", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_reshard_num_logs": "16", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_reshard_progress_judge_interval": "120", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_reshard_thread_interval": "600", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_reshardlog_threshold": "30000", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_resolve_cname": "false", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_rest_getusage_op_compat": "false", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_restore_debug_interval": "-1", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_restore_lock_max_time": "90", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_restore_max_objs": "32", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_restore_processor_period": "900", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_run_sync_thread": "true", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_s3_auth_disable_signature_url": "false", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_s3_auth_order": "sts, external, local", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_s3_auth_use_keystone": "false", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_s3_auth_use_ldap": "false", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_s3_auth_use_rados": "true", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_s3_auth_use_sts": "false", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_s3_client_max_sig_ver": "-1", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_s3_success_create_obj_status": "0", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_safe_max_objects_per_shard": "102400", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_scheduler_type": "throttler", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_script_uri": "", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_service_provider_name": "", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_shard_warning_threshold": "90.000000", 2026-02-25T08:40:43.956 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sts_client_id": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sts_client_secret": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sts_entry": "sts", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sts_key": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sts_max_session_duration": "43200", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sts_min_session_duration": "900", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sts_token_introspection_url": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_swift_account_in_url": "false", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_swift_auth_entry": "auth", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_swift_auth_url": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_swift_custom_header": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_swift_enforce_content_length": "false", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_swift_need_stats": "true", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_swift_tenant_name": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_swift_token_expiration": "86400", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_swift_url": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_swift_url_prefix": "swift", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_swift_versioning_enabled": "false", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_lease_period": "120", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_log_trim_interval": "1200", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_log_trim_max_buckets": "16", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_obj_etag_verify": "false", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_trace_history_size": "4096", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_trace_per_node_log_size": "32", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_thread_pool_size": "512", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_topic_persistency_max_retries": "0", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_topic_persistency_sleep_duration": "0", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_topic_persistency_time_to_live": "0", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_topic_require_publish_policy": "false", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_torrent_comment": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_torrent_createby": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_torrent_encoding": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_torrent_flag": "false", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_torrent_max_size": "5368709120", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_torrent_origin": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_torrent_sha_unit": "524288", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_torrent_tracker": "", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_trust_forwarded_https": "false", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_usage_log_flush_threshold": "1024", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_usage_log_key_transition": "true", 2026-02-25T08:40:43.957 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_usage_log_tick_interval": "30", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_usage_max_shards": "32", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_usage_max_user_shards": "1", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_use_opa_authz": "false", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_user_counters_cache": "false", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_user_counters_cache_size": "10000", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_user_default_quota_max_objects": "-1", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_user_default_quota_max_size": "-1", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_user_max_buckets": "1000", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_user_policies_max_num": "100", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_user_quota_sync_idle_users": "false", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_user_quota_sync_interval": "86400", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_user_quota_sync_wait_time": "86400", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_user_unique_email": "true", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_verify_ssl": "true", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_website_routing_rules_max_num": "50", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_zone": "", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_zone_id": "", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_zone_root_pool": ".rgw.root", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_zonegroup": "", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_zonegroup_id": "", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgwlc_auto_session_clear": "true", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rgwlc_skip_bucket_step": "false", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_block_size": "4096", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_bloom_bits_per_key": "20", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_cache_row_ratio": "0.000000", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_cache_shard_bits": "4", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_cache_size": "536870912", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_cache_type": "binned_lru", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_cf_compact_on_deletion": "true", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_collect_compaction_stats": "false", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_collect_extended_stats": "false", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_collect_memory_stats": "false", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_delete_range_threshold": "1048576", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_index_type": "binary_search", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_log_to_ceph_log": "true", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_metadata_block_size": "4096", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_partition_filters": "false", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_perf": "false", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2026-02-25T08:40:43.958 INFO:teuthology.orchestra.run.trial006.stdout: "rotating_keys_bootstrap_timeout": "30", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "rotating_keys_renewal_timeout": "10", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "run_dir": "/var/run/ceph", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_block_create": "true", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_cachepin_2q_in_ratio": "0.500000", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_cachepin_2q_out_ratio": "0.500000", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_cachepin_size_pershard": "2147483648", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_cachepin_type": "LRU", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_cbjournal_size": "5368709120", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_cold_tier_generations": "3", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_data_delta_based_overwrite": "0", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_default_max_object_size": "16777216", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_default_object_metadata_reservation": "16777216", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_device_size": "53687091200", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_disable_end_to_end_data_protection": "true", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_full_integrity_check": "false", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_hot_tier_generations": "5", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_journal_batch_capacity": "16", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_journal_batch_flush_size": "16777216", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_journal_iodepth_limit": "5", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_main_device_type": "SSD", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_max_concurrent_transactions": "128", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_max_data_allocation_size": "0", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_segment_cleaner_gc_formula": "cost_benefit", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "seastore_segment_size": "67108864", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "service_unique_id": "", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "set_keepcaps": "false", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "setgroup": "", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "setuser": "", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "setuser_match_path": "", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "subv_metrics_window_interval": "30", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "target_max_misplaced_ratio": "0.050000", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "thp": "false", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "threadpool_default_timeout": "60", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "threadpool_empty_queue_max_wait": "2", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "throttler_perf_counter": "true", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "tmp_dir": "/tmp", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "uadk_compressor_enabled": "false", 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout: "uadk_wd_sync_ctx_num": "2" 2026-02-25T08:40:43.959 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-25T08:40:43.961 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2026-02-25T08:40:44.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:44.034+0000 7f05f470b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:44.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:44.037+0000 7f05f470b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:44.047 INFO:teuthology.orchestra.run.trial006.stdout:{} 2026-02-25T08:40:44.048 INFO:teuthology.orchestra.run.trial006.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2026-02-25T08:40:44.054 DEBUG:teuthology.orchestra.run.trial006:> type ceph-objectstore-tool 2026-02-25T08:40:44.070 INFO:teuthology.orchestra.run.trial006.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2026-02-25T08:40:44.070 DEBUG:teuthology.orchestra.run.trial158:> type ceph-objectstore-tool 2026-02-25T08:40:44.086 INFO:teuthology.orchestra.run.trial158.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2026-02-25T08:40:44.086 INFO:teuthology.run_tasks:Running task rados... 2026-02-25T08:40:44.105 INFO:tasks.rados:Beginning rados... 2026-02-25T08:40:44.105 INFO:tasks.rados:config is {'clients': ['client.0'], 'localize_reads': True, 'objects': 50, 'op_weights': {'copy_from': 0, 'delete': 50, 'read': 100, 'rollback': 0, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000} 2026-02-25T08:40:44.105 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-logstoremanager'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd target transaction size': 5, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'Degraded data redundancy', 'is down', 'osds down', 'pg .*? is .*?degraded.*?, acting', 'pg .*? is stuck', 'pg degraded', 'PG_BACKFILL_FULL', 'Low space hindering backfill .*? backfill_toofull', 'OSD_ROOT_DOWN', 'pgs degraded', 'pgs undersized', 'is active.*backfill_toofull.*'], 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'c32469671c6b0222f786dd36c34eb702f780a11d'}} 2026-02-25T08:40:44.105 INFO:tasks.rados:config is {'clients': ['client.0'], 'localize_reads': True, 'objects': 50, 'op_weights': {'copy_from': 0, 'delete': 50, 'read': 100, 'rollback': 0, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000} 2026-02-25T08:40:44.105 DEBUG:teuthology.run_tasks:Unwinding manager rados 2026-02-25T08:40:44.119 INFO:tasks.rados:joining rados 2026-02-25T08:40:44.119 INFO:tasks.thrashosds.thrasher:starting do_thrash 2026-02-25T08:40:44.119 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-25T08:40:44.119 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2026-02-25T08:40:44.120 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:40:44.120 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2026-02-25T08:40:44.120 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:40:44.121 INFO:tasks.rados:clients are ['client.0', 'client.1'] 2026-02-25T08:40:44.121 INFO:tasks.rados:starting run 0 out of 1 2026-02-25T08:40:44.121 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2026-02-25T08:40:44.121 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2026-02-25T08:40:44.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:44.197+0000 7fd8ca7dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:44.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:44.197+0000 7f8af285a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:44.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:44.201+0000 7fd8ca7dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:44.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:44.198+0000 7fdd7c7a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:44.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:44.203+0000 7f8af285a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:44.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:44.205+0000 7fdd7c7a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:44.321 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-25T08:40:44.321 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-25T08:40:44.331 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2026-02-25T08:40:44.000287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":4056,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208662568,"statfs":{"total":213674622976,"available":213670469632,"internally_reserved":0,"allocated":4153344,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.963582"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.043008+0000","last_change":"2026-02-25T08:40:41.045744+0000","last_active":"2026-02-25T08:40:41.043008+0000","last_peered":"2026-02-25T08:40:41.043008+0000","last_clean":"2026-02-25T08:40:41.043008+0000","last_became_active":"2026-02-25T08:40:39.088696+0000","last_became_peered":"2026-02-25T08:40:39.088696+0000","last_unstale":"2026-02-25T08:40:41.043008+0000","last_undegraded":"2026-02-25T08:40:41.043008+0000","last_fullsized":"2026-02-25T08:40:41.043008+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.043049+0000","last_change":"2026-02-25T08:40:41.045214+0000","last_active":"2026-02-25T08:40:41.043049+0000","last_peered":"2026-02-25T08:40:41.043049+0000","last_clean":"2026-02-25T08:40:41.043049+0000","last_became_active":"2026-02-25T08:40:39.071192+0000","last_became_peered":"2026-02-25T08:40:39.071192+0000","last_unstale":"2026-02-25T08:40:41.043049+0000","last_undegraded":"2026-02-25T08:40:41.043049+0000","last_fullsized":"2026-02-25T08:40:41.043049+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040692+0000","last_change":"2026-02-25T08:40:41.042930+0000","last_active":"2026-02-25T08:40:41.040692+0000","last_peered":"2026-02-25T08:40:41.040692+0000","last_clean":"2026-02-25T08:40:41.040692+0000","last_became_active":"2026-02-25T08:40:39.066104+0000","last_became_peered":"2026-02-25T08:40:39.066104+0000","last_unstale":"2026-02-25T08:40:41.040692+0000","last_undegraded":"2026-02-25T08:40:41.040692+0000","last_fullsized":"2026-02-25T08:40:41.040692+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040718+0000","last_change":"2026-02-25T08:40:41.043204+0000","last_active":"2026-02-25T08:40:41.040718+0000","last_peered":"2026-02-25T08:40:41.040718+0000","last_clean":"2026-02-25T08:40:41.040718+0000","last_became_active":"2026-02-25T08:40:39.068832+0000","last_became_peered":"2026-02-25T08:40:39.068832+0000","last_unstale":"2026-02-25T08:40:41.040718+0000","last_undegraded":"2026-02-25T08:40:41.040718+0000","last_fullsized":"2026-02-25T08:40:41.040718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":82,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.042450+0000","last_change":"2026-02-25T08:40:39.100603+0000","last_active":"2026-02-25T08:40:41.042450+0000","last_peered":"2026-02-25T08:40:41.042450+0000","last_clean":"2026-02-25T08:40:41.042450+0000","last_became_active":"2026-02-25T08:40:39.090744+0000","last_became_peered":"2026-02-25T08:40:39.090744+0000","last_unstale":"2026-02-25T08:40:41.042450+0000","last_undegraded":"2026-02-25T08:40:41.042450+0000","last_fullsized":"2026-02-25T08:40:41.042450+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.041023+0000","last_change":"2026-02-25T08:40:41.043569+0000","last_active":"2026-02-25T08:40:41.041023+0000","last_peered":"2026-02-25T08:40:41.041023+0000","last_clean":"2026-02-25T08:40:41.041023+0000","last_became_active":"2026-02-25T08:40:39.072385+0000","last_became_peered":"2026-02-25T08:40:39.072385+0000","last_unstale":"2026-02-25T08:40:41.041023+0000","last_undegraded":"2026-02-25T08:40:41.041023+0000","last_fullsized":"2026-02-25T08:40:41.041023+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.042296+0000","last_change":"2026-02-25T08:40:41.042296+0000","last_active":"2026-02-25T08:40:41.042296+0000","last_peered":"2026-02-25T08:40:41.042296+0000","last_clean":"2026-02-25T08:40:41.042296+0000","last_became_active":"2026-02-25T08:40:39.072423+0000","last_became_peered":"2026-02-25T08:40:39.072423+0000","last_unstale":"2026-02-25T08:40:41.042296+0000","last_undegraded":"2026-02-25T08:40:41.042296+0000","last_fullsized":"2026-02-25T08:40:41.042296+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.040777+0000","last_change":"2026-02-25T08:40:41.043202+0000","last_active":"2026-02-25T08:40:41.040777+0000","last_peered":"2026-02-25T08:40:41.040777+0000","last_clean":"2026-02-25T08:40:41.040777+0000","last_became_active":"2026-02-25T08:40:39.068039+0000","last_became_peered":"2026-02-25T08:40:39.068039+0000","last_unstale":"2026-02-25T08:40:41.040777+0000","last_undegraded":"2026-02-25T08:40:41.040777+0000","last_fullsized":"2026-02-25T08:40:41.040777+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T08:40:41.039223+0000","last_change":"2026-02-25T08:40:41.042120+0000","last_active":"2026-02-25T08:40:41.039223+0000","last_peered":"2026-02-25T08:40:41.039223+0000","last_clean":"2026-02-25T08:40:41.039223+0000","last_became_active":"2026-02-25T08:40:39.085713+0000","last_became_peered":"2026-02-25T08:40:39.085713+0000","last_unstale":"2026-02-25T08:40:41.039223+0000","last_undegraded":"2026-02-25T08:40:41.039223+0000","last_fullsized":"2026-02-25T08:40:41.039223+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:40:38.021390+0000","last_clean_scrub_stamp":"2026-02-25T08:40:38.021390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":213674622976,"available":213670469632,"internally_reserved":0,"allocated":4153344,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":448,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166208,"statfs":{"total":53418655744,"available":53418196992,"internally_reserved":0,"allocated":458752,"data_stored":458752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1204,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165452,"statfs":{"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1204,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165452,"statfs":{"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1200,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165456,"statfs":{"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":53418655744,"available":53417422848,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53418196992,"internally_reserved":0,"allocated":458752,"data_stored":458752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417426944,"internally_reserved":0,"allocated":1228800,"data_stored":1228800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:40:45.186 INFO:teuthology.orchestra.run.trial006.stderr:pool 'unique_pool_0' created 2026-02-25T08:40:45.199 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2026-02-25T08:40:45.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:45.275+0000 7f0e03d74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:45.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:45.283+0000 7f0e03d74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:46.190 INFO:teuthology.orchestra.run.trial006.stderr:enabled application 'rados' on pool 'unique_pool_0' 2026-02-25T08:40:46.268 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:40:46.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:46.343+0000 7f9a606cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:46.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:46.346+0000 7f9a606cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:47.205 DEBUG:teuthology.orchestra.run.trial006:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph_test_rados --localize-reads --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 0 --op copy_from 0 --op write_excl 50 --pool unique_pool_0 2026-02-25T08:40:47.252 INFO:tasks.rados.rados.0.trial006.stdout:adding op weight read -> 100 2026-02-25T08:40:47.252 INFO:tasks.rados.rados.0.trial006.stdout:adding op weight write -> 50 2026-02-25T08:40:47.252 INFO:tasks.rados.rados.0.trial006.stdout:adding op weight delete -> 50 2026-02-25T08:40:47.253 INFO:tasks.rados.rados.0.trial006.stdout:adding op weight snap_create -> 50 2026-02-25T08:40:47.253 INFO:tasks.rados.rados.0.trial006.stdout:adding op weight snap_remove -> 50 2026-02-25T08:40:47.253 INFO:tasks.rados.rados.0.trial006.stdout:adding op weight write_excl -> 50 2026-02-25T08:40:47.253 INFO:tasks.rados.rados.0.trial006.stdout:ceph version 20.3.0-5338-g6aeb6975 (6aeb6975bc9a4790a399c9cc70dba9635d2a7819) tentacle (dev - Debug) 2026-02-25T08:40:47.253 INFO:tasks.rados.rados.0.trial006.stdout:Configuration: 2026-02-25T08:40:47.253 INFO:tasks.rados.rados.0.trial006.stdout: Number of operations: 4000 2026-02-25T08:40:47.253 INFO:tasks.rados.rados.0.trial006.stdout: Number of objects: 50 2026-02-25T08:40:47.253 INFO:tasks.rados.rados.0.trial006.stdout: Max in flight operations: 16 2026-02-25T08:40:47.253 INFO:tasks.rados.rados.0.trial006.stdout: Object size (in bytes): 4000000 2026-02-25T08:40:47.253 INFO:tasks.rados.rados.0.trial006.stdout: Write stride min: 400000 2026-02-25T08:40:47.253 INFO:tasks.rados.rados.0.trial006.stdout: Write stride max: 800000 2026-02-25T08:40:48.197 INFO:tasks.rados.rados.0.trial006.stdout:1: write initial oid 1 2026-02-25T08:40:48.198 INFO:tasks.rados.rados.0.trial006.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2026-02-25T08:40:48.200 INFO:tasks.rados.rados.0.trial006.stdout:1: writing trial00642507-1 from 416807 to 892056 tid 1 2026-02-25T08:40:48.206 INFO:tasks.rados.rados.0.trial006.stdout:1: writing trial00642507-1 from 1542129 to 2016807 tid 2 2026-02-25T08:40:48.206 INFO:tasks.rados.rados.0.trial006.stdout:2: write initial oid 2 2026-02-25T08:40:48.206 INFO:tasks.rados.rados.0.trial006.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2026-02-25T08:40:48.209 INFO:tasks.rados.rados.0.trial006.stdout:2: writing trial00642507-2 from 416807 to 892056 tid 1 2026-02-25T08:40:48.214 INFO:tasks.rados.rados.0.trial006.stdout:2: writing trial00642507-2 from 1542129 to 2016807 tid 2 2026-02-25T08:40:48.214 INFO:tasks.rados.rados.0.trial006.stdout:3: write initial oid 3 2026-02-25T08:40:48.214 INFO:tasks.rados.rados.0.trial006.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2026-02-25T08:40:48.218 INFO:tasks.rados.rados.0.trial006.stdout:3: writing trial00642507-3 from 433614 to 984112 tid 1 2026-02-25T08:40:48.222 INFO:tasks.rados.rados.0.trial006.stdout:3: writing trial00642507-3 from 1600611 to 2033614 tid 2 2026-02-25T08:40:48.222 INFO:tasks.rados.rados.0.trial006.stdout:4: write initial oid 4 2026-02-25T08:40:48.222 INFO:tasks.rados.rados.0.trial006.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2026-02-25T08:40:48.226 INFO:tasks.rados.rados.0.trial006.stdout:4: writing trial00642507-4 from 450421 to 1076168 tid 1 2026-02-25T08:40:48.230 INFO:tasks.rados.rados.0.trial006.stdout:4: writing trial00642507-4 from 1659093 to 2050421 tid 2 2026-02-25T08:40:48.230 INFO:tasks.rados.rados.0.trial006.stdout:5: write initial oid 5 2026-02-25T08:40:48.231 INFO:tasks.rados.rados.0.trial006.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2026-02-25T08:40:48.235 INFO:tasks.rados.rados.0.trial006.stdout:5: writing trial00642507-5 from 467228 to 1168224 tid 1 2026-02-25T08:40:48.238 INFO:tasks.rados.rados.0.trial006.stdout:5: writing trial00642507-5 from 1717575 to 2067228 tid 2 2026-02-25T08:40:48.239 INFO:tasks.rados.rados.0.trial006.stdout:6: write initial oid 6 2026-02-25T08:40:48.239 INFO:tasks.rados.rados.0.trial006.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2026-02-25T08:40:48.243 INFO:tasks.rados.rados.0.trial006.stdout:6: writing trial00642507-6 from 484035 to 1260280 tid 1 2026-02-25T08:40:48.247 INFO:tasks.rados.rados.0.trial006.stdout:6: writing trial00642507-6 from 2059704 to 2084035 tid 2 2026-02-25T08:40:48.247 INFO:tasks.rados.rados.0.trial006.stdout:7: write initial oid 7 2026-02-25T08:40:48.247 INFO:tasks.rados.rados.0.trial006.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2026-02-25T08:40:48.250 INFO:tasks.rados.rados.0.trial006.stdout:7: writing trial00642507-7 from 500842 to 952336 tid 1 2026-02-25T08:40:48.255 INFO:tasks.rados.rados.0.trial006.stdout:7: writing trial00642507-7 from 1718186 to 2100842 tid 2 2026-02-25T08:40:48.255 INFO:tasks.rados.rados.0.trial006.stdout:8: write initial oid 8 2026-02-25T08:40:48.255 INFO:tasks.rados.rados.0.trial006.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2026-02-25T08:40:48.259 INFO:tasks.rados.rados.0.trial006.stdout:8: writing trial00642507-8 from 517649 to 1044392 tid 1 2026-02-25T08:40:48.264 INFO:tasks.rados.rados.0.trial006.stdout:8: writing trial00642507-8 from 1776668 to 2117649 tid 2 2026-02-25T08:40:48.264 INFO:tasks.rados.rados.0.trial006.stdout:9: write initial oid 9 2026-02-25T08:40:48.264 INFO:tasks.rados.rados.0.trial006.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2026-02-25T08:40:48.268 INFO:tasks.rados.rados.0.trial006.stdout:9: writing trial00642507-9 from 534456 to 1252801 tid 1 2026-02-25T08:40:48.272 INFO:tasks.rados.rados.0.trial006.stdout:9: writing trial00642507-9 from 1951503 to 2134456 tid 2 2026-02-25T08:40:48.272 INFO:tasks.rados.rados.0.trial006.stdout:10: write initial oid 10 2026-02-25T08:40:48.272 INFO:tasks.rados.rados.0.trial006.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2026-02-25T08:40:48.277 INFO:tasks.rados.rados.0.trial006.stdout:10: writing trial00642507-10 from 551263 to 1344857 tid 1 2026-02-25T08:40:48.280 INFO:tasks.rados.rados.0.trial006.stdout:10: writing trial00642507-10 from 1893632 to 2151263 tid 2 2026-02-25T08:40:48.280 INFO:tasks.rados.rados.0.trial006.stdout:11: write initial oid 11 2026-02-25T08:40:48.280 INFO:tasks.rados.rados.0.trial006.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2026-02-25T08:40:48.284 INFO:tasks.rados.rados.0.trial006.stdout:11: writing trial00642507-11 from 568070 to 1036913 tid 1 2026-02-25T08:40:48.289 INFO:tasks.rados.rados.0.trial006.stdout:11: writing trial00642507-11 from 1552114 to 2168070 tid 2 2026-02-25T08:40:48.289 INFO:tasks.rados.rados.0.trial006.stdout:12: write initial oid 12 2026-02-25T08:40:48.289 INFO:tasks.rados.rados.0.trial006.stdout:1: finishing write tid 1 to trial00642507-1 2026-02-25T08:40:48.289 INFO:tasks.rados.rados.0.trial006.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2026-02-25T08:40:48.293 INFO:tasks.rados.rados.0.trial006.stdout:12: writing trial00642507-12 from 584877 to 1128969 tid 1 2026-02-25T08:40:48.297 INFO:tasks.rados.rados.0.trial006.stdout:12: writing trial00642507-12 from 1610596 to 2172599 tid 2 2026-02-25T08:40:48.297 INFO:tasks.rados.rados.0.trial006.stdout:13: write initial oid 13 2026-02-25T08:40:48.297 INFO:tasks.rados.rados.0.trial006.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2026-02-25T08:40:48.302 INFO:tasks.rados.rados.0.trial006.stdout:13: writing trial00642507-13 from 601684 to 1221025 tid 1 2026-02-25T08:40:48.306 INFO:tasks.rados.rados.0.trial006.stdout:13: writing trial00642507-13 from 1669078 to 2201684 tid 2 2026-02-25T08:40:48.306 INFO:tasks.rados.rados.0.trial006.stdout:14: write initial oid 14 2026-02-25T08:40:48.306 INFO:tasks.rados.rados.0.trial006.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2026-02-25T08:40:48.311 INFO:tasks.rados.rados.0.trial006.stdout:14: writing trial00642507-14 from 618491 to 1313081 tid 1 2026-02-25T08:40:48.314 INFO:tasks.rados.rados.0.trial006.stdout:14: writing trial00642507-14 from 2011207 to 2218491 tid 2 2026-02-25T08:40:48.315 INFO:tasks.rados.rados.0.trial006.stdout:15: write initial oid 15 2026-02-25T08:40:48.315 INFO:tasks.rados.rados.0.trial006.stdout:2: finishing write tid 1 to trial00642507-2 2026-02-25T08:40:48.315 INFO:tasks.rados.rados.0.trial006.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2026-02-25T08:40:48.320 INFO:tasks.rados.rados.0.trial006.stdout:15: writing trial00642507-15 from 635298 to 1405137 tid 1 2026-02-25T08:40:48.323 INFO:tasks.rados.rados.0.trial006.stdout:15: writing trial00642507-15 from 2069689 to 2235298 tid 2 2026-02-25T08:40:48.323 INFO:tasks.rados.rados.0.trial006.stdout:16: write initial oid 16 2026-02-25T08:40:48.323 INFO:tasks.rados.rados.0.trial006.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2026-02-25T08:40:48.327 INFO:tasks.rados.rados.0.trial006.stdout:16: writing trial00642507-16 from 652105 to 1097193 tid 1 2026-02-25T08:40:48.332 INFO:tasks.rados.rados.0.trial006.stdout:16: writing trial00642507-16 from 1728171 to 2181159 tid 2 2026-02-25T08:40:48.332 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.332 INFO:tasks.rados.rados.0.trial006.stdout:1: finishing write tid 2 to trial00642507-1 2026-02-25T08:40:48.332 INFO:tasks.rados.rados.0.trial006.stdout:3: finishing write tid 1 to trial00642507-3 2026-02-25T08:40:48.332 INFO:tasks.rados.rados.0.trial006.stdout:1: finishing write tid 3 to trial00642507-1 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:1: finishing write tid 4 to trial00642507-1 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:1: oid 1 updating version 0 to 1 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:1: oid 1 updating version 1 to 3 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:1: oid 1 version 3 is already newer than 3 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:5: finishing write tid 1 to trial00642507-5 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:3: finishing write tid 2 to trial00642507-3 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:3: finishing write tid 3 to trial00642507-3 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:3: finishing write tid 4 to trial00642507-3 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:3: oid 3 updating version 0 to 1 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:3: oid 3 updating version 1 to 3 2026-02-25T08:40:48.333 INFO:tasks.rados.rados.0.trial006.stdout:3: oid 3 version 3 is already newer than 3 2026-02-25T08:40:48.334 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2026-02-25T08:40:48.334 INFO:tasks.rados.rados.0.trial006.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2026-02-25T08:40:48.334 INFO:tasks.rados.rados.0.trial006.stdout:4: finishing write tid 1 to trial00642507-4 2026-02-25T08:40:48.334 INFO:tasks.rados.rados.0.trial006.stdout:1: done (15 left) 2026-02-25T08:40:48.334 INFO:tasks.rados.rados.0.trial006.stdout:3: done (14 left) 2026-02-25T08:40:48.334 INFO:tasks.rados.rados.0.trial006.stdout:17: write initial oid 17 2026-02-25T08:40:48.334 INFO:tasks.rados.rados.0.trial006.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2026-02-25T08:40:48.336 INFO:tasks.rados.rados.0.trial006.stdout:17: writing trial00642507-17 from 668912 to 1305602 tid 1 2026-02-25T08:40:48.340 INFO:tasks.rados.rados.0.trial006.stdout:17: writing trial00642507-17 from 1903006 to 2268912 tid 2 2026-02-25T08:40:48.340 INFO:tasks.rados.rados.0.trial006.stdout:18: write initial oid 18 2026-02-25T08:40:48.340 INFO:tasks.rados.rados.0.trial006.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2026-02-25T08:40:48.345 INFO:tasks.rados.rados.0.trial006.stdout:18: writing trial00642507-18 from 685719 to 1397658 tid 1 2026-02-25T08:40:48.349 INFO:tasks.rados.rados.0.trial006.stdout:18: writing trial00642507-18 from 1845135 to 2285719 tid 2 2026-02-25T08:40:48.350 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.350 INFO:tasks.rados.rados.0.trial006.stdout:7: finishing write tid 1 to trial00642507-7 2026-02-25T08:40:48.350 INFO:tasks.rados.rados.0.trial006.stdout:5: finishing write tid 2 to trial00642507-5 2026-02-25T08:40:48.350 INFO:tasks.rados.rados.0.trial006.stdout:4: finishing write tid 2 to trial00642507-4 2026-02-25T08:40:48.350 INFO:tasks.rados.rados.0.trial006.stdout:5: finishing write tid 3 to trial00642507-5 2026-02-25T08:40:48.350 INFO:tasks.rados.rados.0.trial006.stdout:4: finishing write tid 3 to trial00642507-4 2026-02-25T08:40:48.350 INFO:tasks.rados.rados.0.trial006.stdout:5: finishing write tid 4 to trial00642507-5 2026-02-25T08:40:48.350 INFO:tasks.rados.rados.0.trial006.stdout:5: oid 5 updating version 0 to 4 2026-02-25T08:40:48.350 INFO:tasks.rados.rados.0.trial006.stdout:5: oid 5 updating version 4 to 6 2026-02-25T08:40:48.350 INFO:tasks.rados.rados.0.trial006.stdout:5: oid 5 version 6 is already newer than 6 2026-02-25T08:40:48.350 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 5 v 6 (ObjNum 4 snap 0 seq_num 4) dirty exists 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:4: finishing write tid 4 to trial00642507-4 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:4: oid 4 updating version 0 to 6 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:4: oid 4 version 6 is already newer than 4 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:4: oid 4 version 6 is already newer than 6 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 4 v 6 (ObjNum 3 snap 0 seq_num 3) dirty exists 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:7: finishing write tid 2 to trial00642507-7 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:7: finishing write tid 3 to trial00642507-7 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:7: finishing write tid 4 to trial00642507-7 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:7: oid 7 updating version 0 to 3 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:7: oid 7 version 3 is already newer than 1 2026-02-25T08:40:48.351 INFO:tasks.rados.rados.0.trial006.stdout:7: oid 7 version 3 is already newer than 3 2026-02-25T08:40:48.352 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 7 v 3 (ObjNum 6 snap 0 seq_num 6) dirty exists 2026-02-25T08:40:48.352 INFO:tasks.rados.rados.0.trial006.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2026-02-25T08:40:48.352 INFO:tasks.rados.rados.0.trial006.stdout:9: finishing write tid 1 to trial00642507-9 2026-02-25T08:40:48.352 INFO:tasks.rados.rados.0.trial006.stdout:10: finishing write tid 1 to trial00642507-10 2026-02-25T08:40:48.352 INFO:tasks.rados.rados.0.trial006.stdout:4: done (15 left) 2026-02-25T08:40:48.352 INFO:tasks.rados.rados.0.trial006.stdout:5: done (14 left) 2026-02-25T08:40:48.352 INFO:tasks.rados.rados.0.trial006.stdout:7: done (13 left) 2026-02-25T08:40:48.352 INFO:tasks.rados.rados.0.trial006.stdout:19: write initial oid 19 2026-02-25T08:40:48.352 INFO:tasks.rados.rados.0.trial006.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2026-02-25T08:40:48.355 INFO:tasks.rados.rados.0.trial006.stdout:19: writing trial00642507-19 from 702526 to 1489714 tid 1 2026-02-25T08:40:48.358 INFO:tasks.rados.rados.0.trial006.stdout:19: writing trial00642507-19 from 1903617 to 2302526 tid 2 2026-02-25T08:40:48.358 INFO:tasks.rados.rados.0.trial006.stdout:20: write initial oid 20 2026-02-25T08:40:48.358 INFO:tasks.rados.rados.0.trial006.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2026-02-25T08:40:48.363 INFO:tasks.rados.rados.0.trial006.stdout:20: writing trial00642507-20 from 719333 to 1181770 tid 1 2026-02-25T08:40:48.367 INFO:tasks.rados.rados.0.trial006.stdout:20: writing trial00642507-20 from 1962099 to 2319333 tid 2 2026-02-25T08:40:48.367 INFO:tasks.rados.rados.0.trial006.stdout:21: write initial oid 21 2026-02-25T08:40:48.368 INFO:tasks.rados.rados.0.trial006.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2026-02-25T08:40:48.373 INFO:tasks.rados.rados.0.trial006.stdout:21: writing trial00642507-21 from 736140 to 1273826 tid 1 2026-02-25T08:40:48.377 INFO:tasks.rados.rados.0.trial006.stdout:21: writing trial00642507-21 from 2020581 to 2336140 tid 2 2026-02-25T08:40:48.377 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.378 INFO:tasks.rados.rados.0.trial006.stdout:2: finishing write tid 2 to trial00642507-2 2026-02-25T08:40:48.378 INFO:tasks.rados.rados.0.trial006.stdout:2: finishing write tid 3 to trial00642507-2 2026-02-25T08:40:48.378 INFO:tasks.rados.rados.0.trial006.stdout:2: finishing write tid 4 to trial00642507-2 2026-02-25T08:40:48.378 INFO:tasks.rados.rados.0.trial006.stdout:2: oid 2 updating version 0 to 3 2026-02-25T08:40:48.378 INFO:tasks.rados.rados.0.trial006.stdout:2: oid 2 version 3 is already newer than 1 2026-02-25T08:40:48.378 INFO:tasks.rados.rados.0.trial006.stdout:2: oid 2 version 3 is already newer than 3 2026-02-25T08:40:48.378 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2026-02-25T08:40:48.378 INFO:tasks.rados.rados.0.trial006.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2026-02-25T08:40:48.378 INFO:tasks.rados.rados.0.trial006.stdout:13: finishing write tid 1 to trial00642507-13 2026-02-25T08:40:48.378 INFO:tasks.rados.rados.0.trial006.stdout:10: finishing write tid 2 to trial00642507-10 2026-02-25T08:40:48.378 INFO:tasks.rados.rados.0.trial006.stdout:9: finishing write tid 2 to trial00642507-9 2026-02-25T08:40:48.379 INFO:tasks.rados.rados.0.trial006.stdout:8: finishing write tid 1 to trial00642507-8 2026-02-25T08:40:48.379 INFO:tasks.rados.rados.0.trial006.stdout:9: finishing write tid 3 to trial00642507-9 2026-02-25T08:40:48.379 INFO:tasks.rados.rados.0.trial006.stdout:2: done (15 left) 2026-02-25T08:40:48.379 INFO:tasks.rados.rados.0.trial006.stdout:22: write initial oid 22 2026-02-25T08:40:48.379 INFO:tasks.rados.rados.0.trial006.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2026-02-25T08:40:48.383 INFO:tasks.rados.rados.0.trial006.stdout:22: writing trial00642507-22 from 752947 to 1365882 tid 1 2026-02-25T08:40:48.390 INFO:tasks.rados.rados.0.trial006.stdout:22: writing trial00642507-22 from 1962710 to 2352947 tid 2 2026-02-25T08:40:48.390 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.390 INFO:tasks.rados.rados.0.trial006.stdout:10: finishing write tid 3 to trial00642507-10 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:10: finishing write tid 4 to trial00642507-10 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:10: oid 10 updating version 0 to 1 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:10: oid 10 updating version 1 to 3 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:10: oid 10 version 3 is already newer than 3 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 10 v 3 (ObjNum 9 snap 0 seq_num 9) dirty exists 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:9: finishing write tid 4 to trial00642507-9 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:9: oid 9 updating version 0 to 9 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:9: oid 9 version 9 is already newer than 9 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:9: oid 9 version 9 is already newer than 7 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 9 v 9 (ObjNum 8 snap 0 seq_num 8) dirty exists 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:9: done (15 left) 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:10: done (14 left) 2026-02-25T08:40:48.391 INFO:tasks.rados.rados.0.trial006.stdout:23: write initial oid 23 2026-02-25T08:40:48.392 INFO:tasks.rados.rados.0.trial006.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2026-02-25T08:40:48.392 INFO:tasks.rados.rados.0.trial006.stdout:23: writing trial00642507-23 from 769754 to 1457938 tid 1 2026-02-25T08:40:48.396 INFO:tasks.rados.rados.0.trial006.stdout:23: writing trial00642507-23 from 2021192 to 2369754 tid 2 2026-02-25T08:40:48.396 INFO:tasks.rados.rados.0.trial006.stdout:24: write initial oid 24 2026-02-25T08:40:48.396 INFO:tasks.rados.rados.0.trial006.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2026-02-25T08:40:48.401 INFO:tasks.rados.rados.0.trial006.stdout:24: writing trial00642507-24 from 786561 to 1266347 tid 1 2026-02-25T08:40:48.406 INFO:tasks.rados.rados.0.trial006.stdout:24: writing trial00642507-24 from 1796027 to 2263691 tid 2 2026-02-25T08:40:48.406 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.406 INFO:tasks.rados.rados.0.trial006.stdout:14: finishing write tid 1 to trial00642507-14 2026-02-25T08:40:48.406 INFO:tasks.rados.rados.0.trial006.stdout:13: finishing write tid 2 to trial00642507-13 2026-02-25T08:40:48.406 INFO:tasks.rados.rados.0.trial006.stdout:12: finishing write tid 1 to trial00642507-12 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:6: finishing write tid 1 to trial00642507-6 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:13: finishing write tid 3 to trial00642507-13 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:13: finishing write tid 4 to trial00642507-13 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:13: oid 13 updating version 0 to 3 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:13: oid 13 version 3 is already newer than 1 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:13: oid 13 version 3 is already newer than 3 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 13 v 3 (ObjNum 12 snap 0 seq_num 12) dirty exists 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:8: finishing write tid 2 to trial00642507-8 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:17: finishing write tid 1 to trial00642507-17 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:13: done (15 left) 2026-02-25T08:40:48.407 INFO:tasks.rados.rados.0.trial006.stdout:25: write initial oid 25 2026-02-25T08:40:48.408 INFO:tasks.rados.rados.0.trial006.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2026-02-25T08:40:48.409 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:40:48.411 INFO:tasks.rados.rados.0.trial006.stdout:25: writing trial00642507-25 from 403368 to 958403 tid 1 2026-02-25T08:40:48.415 INFO:tasks.rados.rados.0.trial006.stdout:25: writing trial00642507-25 from 1454509 to 2182184 tid 2 2026-02-25T08:40:48.415 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:8: finishing write tid 3 to trial00642507-8 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:8: finishing write tid 4 to trial00642507-8 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:8: oid 8 updating version 0 to 1 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:8: oid 8 updating version 1 to 3 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:8: oid 8 version 3 is already newer than 3 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 8 v 3 (ObjNum 7 snap 0 seq_num 7) dirty exists 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:12: finishing write tid 2 to trial00642507-12 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:6: finishing write tid 2 to trial00642507-6 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:17: finishing write tid 2 to trial00642507-17 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:8: done (15 left) 2026-02-25T08:40:48.416 INFO:tasks.rados.rados.0.trial006.stdout:26: write initial oid 26 2026-02-25T08:40:48.417 INFO:tasks.rados.rados.0.trial006.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2026-02-25T08:40:48.418 INFO:tasks.rados.rados.0.trial006.stdout:26: writing trial00642507-26 from 420175 to 1050459 tid 1 2026-02-25T08:40:48.424 INFO:tasks.rados.rados.0.trial006.stdout:26: writing trial00642507-26 from 1796638 to 2267971 tid 2 2026-02-25T08:40:48.424 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.424 INFO:tasks.rados.rados.0.trial006.stdout:17: finishing write tid 3 to trial00642507-17 2026-02-25T08:40:48.424 INFO:tasks.rados.rados.0.trial006.stdout:17: finishing write tid 4 to trial00642507-17 2026-02-25T08:40:48.424 INFO:tasks.rados.rados.0.trial006.stdout:17: oid 17 updating version 0 to 12 2026-02-25T08:40:48.424 INFO:tasks.rados.rados.0.trial006.stdout:17: oid 17 version 12 is already newer than 10 2026-02-25T08:40:48.424 INFO:tasks.rados.rados.0.trial006.stdout:17: oid 17 version 12 is already newer than 12 2026-02-25T08:40:48.424 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 17 v 12 (ObjNum 16 snap 0 seq_num 16) dirty exists 2026-02-25T08:40:48.424 INFO:tasks.rados.rados.0.trial006.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2026-02-25T08:40:48.424 INFO:tasks.rados.rados.0.trial006.stdout:6: finishing write tid 3 to trial00642507-6 2026-02-25T08:40:48.424 INFO:tasks.rados.rados.0.trial006.stdout:12: finishing write tid 3 to trial00642507-12 2026-02-25T08:40:48.425 INFO:tasks.rados.rados.0.trial006.stdout:17: done (15 left) 2026-02-25T08:40:48.425 INFO:tasks.rados.rados.0.trial006.stdout:27: write initial oid 27 2026-02-25T08:40:48.425 INFO:tasks.rados.rados.0.trial006.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2026-02-25T08:40:48.428 INFO:tasks.rados.rados.0.trial006.stdout:27: writing trial00642507-27 from 436982 to 1142515 tid 1 2026-02-25T08:40:48.434 INFO:tasks.rados.rados.0.trial006.stdout:27: writing trial00642507-27 from 1855120 to 2436982 tid 2 2026-02-25T08:40:48.434 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.434 INFO:tasks.rados.rados.0.trial006.stdout:14: finishing write tid 2 to trial00642507-14 2026-02-25T08:40:48.434 INFO:tasks.rados.rados.0.trial006.stdout:20: finishing write tid 1 to trial00642507-20 2026-02-25T08:40:48.434 INFO:tasks.rados.rados.0.trial006.stdout:6: finishing write tid 4 to trial00642507-6 2026-02-25T08:40:48.434 INFO:tasks.rados.rados.0.trial006.stdout:6: oid 6 updating version 0 to 3 2026-02-25T08:40:48.434 INFO:tasks.rados.rados.0.trial006.stdout:6: oid 6 version 3 is already newer than 3 2026-02-25T08:40:48.434 INFO:tasks.rados.rados.0.trial006.stdout:6: oid 6 version 3 is already newer than 1 2026-02-25T08:40:48.434 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:12: finishing write tid 4 to trial00642507-12 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:12: oid 12 updating version 0 to 6 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:12: oid 12 version 6 is already newer than 4 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:12: oid 12 version 6 is already newer than 6 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:21: finishing write tid 1 to trial00642507-21 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:11: finishing write tid 1 to trial00642507-11 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:14: finishing write tid 3 to trial00642507-14 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:18: finishing write tid 1 to trial00642507-18 2026-02-25T08:40:48.435 INFO:tasks.rados.rados.0.trial006.stdout:6: done (15 left) 2026-02-25T08:40:48.436 INFO:tasks.rados.rados.0.trial006.stdout:12: done (14 left) 2026-02-25T08:40:48.436 INFO:tasks.rados.rados.0.trial006.stdout:28: write initial oid 28 2026-02-25T08:40:48.436 INFO:tasks.rados.rados.0.trial006.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2026-02-25T08:40:48.438 INFO:tasks.rados.rados.0.trial006.stdout:28: writing trial00642507-28 from 453789 to 1234571 tid 1 2026-02-25T08:40:48.443 INFO:tasks.rados.rados.0.trial006.stdout:28: writing trial00642507-28 from 1913602 to 2388604 tid 2 2026-02-25T08:40:48.443 INFO:tasks.rados.rados.0.trial006.stdout:29: write initial oid 29 2026-02-25T08:40:48.443 INFO:tasks.rados.rados.0.trial006.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2026-02-25T08:40:48.446 INFO:tasks.rados.rados.0.trial006.stdout:29: writing trial00642507-29 from 470596 to 926627 tid 1 2026-02-25T08:40:48.451 INFO:tasks.rados.rados.0.trial006.stdout:29: writing trial00642507-29 from 1572084 to 2190744 tid 2 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout:21: finishing write tid 2 to trial00642507-21 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout:20: finishing write tid 2 to trial00642507-20 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout:14: finishing write tid 4 to trial00642507-14 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout:14: oid 14 updating version 0 to 6 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout:14: oid 14 version 6 is already newer than 6 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout:14: oid 14 version 6 is already newer than 4 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout:20: finishing write tid 3 to trial00642507-20 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout:11: finishing write tid 2 to trial00642507-11 2026-02-25T08:40:48.452 INFO:tasks.rados.rados.0.trial006.stdout:14: done (15 left) 2026-02-25T08:40:48.453 INFO:tasks.rados.rados.0.trial006.stdout:30: write initial oid 30 2026-02-25T08:40:48.453 INFO:tasks.rados.rados.0.trial006.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2026-02-25T08:40:48.455 INFO:tasks.rados.rados.0.trial006.stdout:30: writing trial00642507-30 from 487403 to 1018683 tid 1 2026-02-25T08:40:48.460 INFO:tasks.rados.rados.0.trial006.stdout:30: writing trial00642507-30 from 1514213 to 1992884 tid 2 2026-02-25T08:40:48.460 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.460 INFO:tasks.rados.rados.0.trial006.stdout:21: finishing write tid 3 to trial00642507-21 2026-02-25T08:40:48.460 INFO:tasks.rados.rados.0.trial006.stdout:20: finishing write tid 4 to trial00642507-20 2026-02-25T08:40:48.460 INFO:tasks.rados.rados.0.trial006.stdout:20: oid 20 updating version 0 to 3 2026-02-25T08:40:48.460 INFO:tasks.rados.rados.0.trial006.stdout:20: oid 20 version 3 is already newer than 1 2026-02-25T08:40:48.460 INFO:tasks.rados.rados.0.trial006.stdout:20: oid 20 version 3 is already newer than 3 2026-02-25T08:40:48.460 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 20 v 3 (ObjNum 19 snap 0 seq_num 19) dirty exists 2026-02-25T08:40:48.460 INFO:tasks.rados.rados.0.trial006.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2026-02-25T08:40:48.460 INFO:tasks.rados.rados.0.trial006.stdout:21: finishing write tid 4 to trial00642507-21 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:21: oid 21 updating version 0 to 3 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:21: oid 21 version 3 is already newer than 3 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:21: oid 21 version 3 is already newer than 1 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 21 v 3 (ObjNum 20 snap 0 seq_num 20) dirty exists 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:11: finishing write tid 3 to trial00642507-11 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:16: finishing write tid 1 to trial00642507-16 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:11: finishing write tid 4 to trial00642507-11 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:11: oid 11 updating version 0 to 6 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:11: oid 11 version 6 is already newer than 4 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:11: oid 11 version 6 is already newer than 6 2026-02-25T08:40:48.461 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 11 v 6 (ObjNum 10 snap 0 seq_num 10) dirty exists 2026-02-25T08:40:48.462 INFO:tasks.rados.rados.0.trial006.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2026-02-25T08:40:48.462 INFO:tasks.rados.rados.0.trial006.stdout:23: finishing write tid 1 to trial00642507-23 2026-02-25T08:40:48.462 INFO:tasks.rados.rados.0.trial006.stdout:11: done (15 left) 2026-02-25T08:40:48.462 INFO:tasks.rados.rados.0.trial006.stdout:20: done (14 left) 2026-02-25T08:40:48.462 INFO:tasks.rados.rados.0.trial006.stdout:21: done (13 left) 2026-02-25T08:40:48.462 INFO:tasks.rados.rados.0.trial006.stdout:31: write initial oid 31 2026-02-25T08:40:48.462 INFO:tasks.rados.rados.0.trial006.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2026-02-25T08:40:48.463 INFO:tasks.rados.rados.0.trial006.stdout:31: writing trial00642507-31 from 504210 to 1110739 tid 1 2026-02-25T08:40:48.468 INFO:tasks.rados.rados.0.trial006.stdout:31: writing trial00642507-31 from 1572695 to 2195024 tid 2 2026-02-25T08:40:48.468 INFO:tasks.rados.rados.0.trial006.stdout:32: write initial oid 32 2026-02-25T08:40:48.468 INFO:tasks.rados.rados.0.trial006.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2026-02-25T08:40:48.473 INFO:tasks.rados.rados.0.trial006.stdout:32: writing trial00642507-32 from 521017 to 1319148 tid 1 2026-02-25T08:40:48.476 INFO:tasks.rados.rados.0.trial006.stdout:32: writing trial00642507-32 from 1747530 to 2229870 tid 2 2026-02-25T08:40:48.476 INFO:tasks.rados.rados.0.trial006.stdout:33: write initial oid 33 2026-02-25T08:40:48.477 INFO:tasks.rados.rados.0.trial006.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2026-02-25T08:40:48.480 INFO:tasks.rados.rados.0.trial006.stdout:33: writing trial00642507-33 from 537824 to 1011204 tid 1 2026-02-25T08:40:48.486 INFO:tasks.rados.rados.0.trial006.stdout:33: writing trial00642507-33 from 1806012 to 2432010 tid 2 2026-02-25T08:40:48.486 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.486 INFO:tasks.rados.rados.0.trial006.stdout:18: finishing write tid 2 to trial00642507-18 2026-02-25T08:40:48.486 INFO:tasks.rados.rados.0.trial006.stdout:18: finishing write tid 3 to trial00642507-18 2026-02-25T08:40:48.486 INFO:tasks.rados.rados.0.trial006.stdout:16: finishing write tid 2 to trial00642507-16 2026-02-25T08:40:48.486 INFO:tasks.rados.rados.0.trial006.stdout:18: finishing write tid 4 to trial00642507-18 2026-02-25T08:40:48.486 INFO:tasks.rados.rados.0.trial006.stdout:18: oid 18 updating version 0 to 1 2026-02-25T08:40:48.486 INFO:tasks.rados.rados.0.trial006.stdout:18: oid 18 updating version 1 to 3 2026-02-25T08:40:48.486 INFO:tasks.rados.rados.0.trial006.stdout:18: oid 18 version 3 is already newer than 3 2026-02-25T08:40:48.487 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 18 v 3 (ObjNum 17 snap 0 seq_num 17) dirty exists 2026-02-25T08:40:48.487 INFO:tasks.rados.rados.0.trial006.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2026-02-25T08:40:48.487 INFO:tasks.rados.rados.0.trial006.stdout:23: finishing write tid 2 to trial00642507-23 2026-02-25T08:40:48.487 INFO:tasks.rados.rados.0.trial006.stdout:18: done (15 left) 2026-02-25T08:40:48.487 INFO:tasks.rados.rados.0.trial006.stdout:34: write initial oid 34 2026-02-25T08:40:48.487 INFO:tasks.rados.rados.0.trial006.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2026-02-25T08:40:48.490 INFO:tasks.rados.rados.0.trial006.stdout:34: writing trial00642507-34 from 554631 to 1103260 tid 1 2026-02-25T08:40:48.492 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:48.488+0000 7fc9d69b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:48.495 INFO:tasks.rados.rados.0.trial006.stdout:34: writing trial00642507-34 from 1748141 to 2234150 tid 2 2026-02-25T08:40:48.495 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.495 INFO:tasks.rados.rados.0.trial006.stdout:23: finishing write tid 3 to trial00642507-23 2026-02-25T08:40:48.495 INFO:tasks.rados.rados.0.trial006.stdout:23: finishing write tid 4 to trial00642507-23 2026-02-25T08:40:48.495 INFO:tasks.rados.rados.0.trial006.stdout:23: oid 23 updating version 0 to 9 2026-02-25T08:40:48.495 INFO:tasks.rados.rados.0.trial006.stdout:23: oid 23 version 9 is already newer than 9 2026-02-25T08:40:48.495 INFO:tasks.rados.rados.0.trial006.stdout:23: oid 23 version 9 is already newer than 7 2026-02-25T08:40:48.495 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 23 v 9 (ObjNum 22 snap 0 seq_num 22) dirty exists 2026-02-25T08:40:48.495 INFO:tasks.rados.rados.0.trial006.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2026-02-25T08:40:48.495 INFO:tasks.rados.rados.0.trial006.stdout:16: finishing write tid 3 to trial00642507-16 2026-02-25T08:40:48.495 INFO:tasks.rados.rados.0.trial006.stdout:23: done (15 left) 2026-02-25T08:40:48.496 INFO:tasks.rados.rados.0.trial006.stdout:35: write initial oid 35 2026-02-25T08:40:48.496 INFO:tasks.rados.rados.0.trial006.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2026-02-25T08:40:48.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:48.493+0000 7fc9d69b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:48.498 INFO:tasks.rados.rados.0.trial006.stdout:35: writing trial00642507-35 from 571438 to 1195316 tid 1 2026-02-25T08:40:48.504 INFO:tasks.rados.rados.0.trial006.stdout:35: writing trial00642507-35 from 1806623 to 2436290 tid 2 2026-02-25T08:40:48.504 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.504 INFO:tasks.rados.rados.0.trial006.stdout:29: finishing write tid 1 to trial00642507-29 2026-02-25T08:40:48.504 INFO:tasks.rados.rados.0.trial006.stdout:16: finishing write tid 4 to trial00642507-16 2026-02-25T08:40:48.504 INFO:tasks.rados.rados.0.trial006.stdout:16: oid 16 updating version 0 to 6 2026-02-25T08:40:48.504 INFO:tasks.rados.rados.0.trial006.stdout:16: oid 16 version 6 is already newer than 6 2026-02-25T08:40:48.504 INFO:tasks.rados.rados.0.trial006.stdout:16: oid 16 version 6 is already newer than 4 2026-02-25T08:40:48.504 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 16 v 6 (ObjNum 15 snap 0 seq_num 15) dirty exists 2026-02-25T08:40:48.504 INFO:tasks.rados.rados.0.trial006.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2026-02-25T08:40:48.505 INFO:tasks.rados.rados.0.trial006.stdout:24: finishing write tid 1 to trial00642507-24 2026-02-25T08:40:48.505 INFO:tasks.rados.rados.0.trial006.stdout:16: done (15 left) 2026-02-25T08:40:48.505 INFO:tasks.rados.rados.0.trial006.stdout:36: write initial oid 36 2026-02-25T08:40:48.505 INFO:tasks.rados.rados.0.trial006.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2026-02-25T08:40:48.508 INFO:tasks.rados.rados.0.trial006.stdout:36: writing trial00642507-36 from 588245 to 1287372 tid 1 2026-02-25T08:40:48.513 INFO:tasks.rados.rados.0.trial006.stdout:36: writing trial00642507-36 from 1865105 to 2354783 tid 2 2026-02-25T08:40:48.513 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:30: finishing write tid 1 to trial00642507-30 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:29: finishing write tid 2 to trial00642507-29 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:31: finishing write tid 1 to trial00642507-31 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:29: finishing write tid 3 to trial00642507-29 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:30: finishing write tid 2 to trial00642507-30 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:30: finishing write tid 3 to trial00642507-30 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:30: finishing write tid 4 to trial00642507-30 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:30: oid 30 updating version 0 to 12 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:30: oid 30 version 12 is already newer than 10 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:30: oid 30 version 12 is already newer than 12 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 30 v 12 (ObjNum 29 snap 0 seq_num 29) dirty exists 2026-02-25T08:40:48.514 INFO:tasks.rados.rados.0.trial006.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2026-02-25T08:40:48.515 INFO:tasks.rados.rados.0.trial006.stdout:31: finishing write tid 2 to trial00642507-31 2026-02-25T08:40:48.515 INFO:tasks.rados.rados.0.trial006.stdout:30: done (15 left) 2026-02-25T08:40:48.515 INFO:tasks.rados.rados.0.trial006.stdout:37: write initial oid 37 2026-02-25T08:40:48.515 INFO:tasks.rados.rados.0.trial006.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2026-02-25T08:40:48.518 INFO:tasks.rados.rados.0.trial006.stdout:37: writing trial00642507-37 from 605052 to 1379428 tid 1 2026-02-25T08:40:48.524 INFO:tasks.rados.rados.0.trial006.stdout:37: writing trial00642507-37 from 1923587 to 2556923 tid 2 2026-02-25T08:40:48.524 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.524 INFO:tasks.rados.rados.0.trial006.stdout:29: finishing write tid 4 to trial00642507-29 2026-02-25T08:40:48.524 INFO:tasks.rados.rados.0.trial006.stdout:29: oid 29 updating version 0 to 9 2026-02-25T08:40:48.524 INFO:tasks.rados.rados.0.trial006.stdout:29: oid 29 version 9 is already newer than 7 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:29: oid 29 version 9 is already newer than 9 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 29 v 9 (ObjNum 28 snap 0 seq_num 28) dirty exists 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:22: finishing write tid 1 to trial00642507-22 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:31: finishing write tid 3 to trial00642507-31 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:31: finishing write tid 4 to trial00642507-31 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:31: oid 31 updating version 0 to 3 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:31: oid 31 version 3 is already newer than 3 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:31: oid 31 version 3 is already newer than 1 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 31 v 3 (ObjNum 30 snap 0 seq_num 30) dirty exists 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:24: finishing write tid 2 to trial00642507-24 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:35: finishing write tid 1 to trial00642507-35 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:29: done (15 left) 2026-02-25T08:40:48.525 INFO:tasks.rados.rados.0.trial006.stdout:31: done (14 left) 2026-02-25T08:40:48.526 INFO:tasks.rados.rados.0.trial006.stdout:38: write initial oid 38 2026-02-25T08:40:48.526 INFO:tasks.rados.rados.0.trial006.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2026-02-25T08:40:48.527 INFO:tasks.rados.rados.0.trial006.stdout:38: writing trial00642507-38 from 621859 to 1071484 tid 1 2026-02-25T08:40:48.533 INFO:tasks.rados.rados.0.trial006.stdout:38: writing trial00642507-38 from 1865716 to 2621859 tid 2 2026-02-25T08:40:48.533 INFO:tasks.rados.rados.0.trial006.stdout:39: write initial oid 39 2026-02-25T08:40:48.533 INFO:tasks.rados.rados.0.trial006.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2026-02-25T08:40:48.538 INFO:tasks.rados.rados.0.trial006.stdout:39: writing trial00642507-39 from 638666 to 1163540 tid 1 2026-02-25T08:40:48.543 INFO:tasks.rados.rados.0.trial006.stdout:39: writing trial00642507-39 from 1924198 to 2561203 tid 2 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout:34: finishing write tid 1 to trial00642507-34 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout:24: finishing write tid 3 to trial00642507-24 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout:24: finishing write tid 4 to trial00642507-24 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout:24: oid 24 updating version 0 to 7 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout:24: oid 24 updating version 7 to 9 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout:24: oid 24 version 9 is already newer than 9 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 24 v 9 (ObjNum 23 snap 0 seq_num 23) dirty exists 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout:35: finishing write tid 2 to trial00642507-35 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout:36: finishing write tid 1 to trial00642507-36 2026-02-25T08:40:48.544 INFO:tasks.rados.rados.0.trial006.stdout:22: finishing write tid 2 to trial00642507-22 2026-02-25T08:40:48.545 INFO:tasks.rados.rados.0.trial006.stdout:34: finishing write tid 2 to trial00642507-34 2026-02-25T08:40:48.545 INFO:tasks.rados.rados.0.trial006.stdout:24: done (15 left) 2026-02-25T08:40:48.545 INFO:tasks.rados.rados.0.trial006.stdout:40: write initial oid 40 2026-02-25T08:40:48.545 INFO:tasks.rados.rados.0.trial006.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2026-02-25T08:40:48.549 INFO:tasks.rados.rados.0.trial006.stdout:40: writing trial00642507-40 from 655473 to 1371949 tid 1 2026-02-25T08:40:48.554 INFO:tasks.rados.rados.0.trial006.stdout:40: writing trial00642507-40 from 2099033 to 2655473 tid 2 2026-02-25T08:40:48.554 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.554 INFO:tasks.rados.rados.0.trial006.stdout:37: finishing write tid 1 to trial00642507-37 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:35: finishing write tid 3 to trial00642507-35 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:22: finishing write tid 3 to trial00642507-22 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:35: finishing write tid 4 to trial00642507-35 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:35: oid 35 updating version 0 to 15 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:35: oid 35 version 15 is already newer than 15 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:35: oid 35 version 15 is already newer than 13 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 35 v 15 (ObjNum 34 snap 0 seq_num 34) dirty exists 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:34: finishing write tid 3 to trial00642507-34 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:22: finishing write tid 4 to trial00642507-22 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:22: oid 22 updating version 0 to 9 2026-02-25T08:40:48.555 INFO:tasks.rados.rados.0.trial006.stdout:22: oid 22 version 9 is already newer than 9 2026-02-25T08:40:48.556 INFO:tasks.rados.rados.0.trial006.stdout:22: oid 22 version 9 is already newer than 7 2026-02-25T08:40:48.556 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 22 v 9 (ObjNum 21 snap 0 seq_num 21) dirty exists 2026-02-25T08:40:48.556 INFO:tasks.rados.rados.0.trial006.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2026-02-25T08:40:48.556 INFO:tasks.rados.rados.0.trial006.stdout:33: finishing write tid 1 to trial00642507-33 2026-02-25T08:40:48.556 INFO:tasks.rados.rados.0.trial006.stdout:22: done (15 left) 2026-02-25T08:40:48.556 INFO:tasks.rados.rados.0.trial006.stdout:35: done (14 left) 2026-02-25T08:40:48.556 INFO:tasks.rados.rados.0.trial006.stdout:41: write initial oid 41 2026-02-25T08:40:48.556 INFO:tasks.rados.rados.0.trial006.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2026-02-25T08:40:48.560 INFO:tasks.rados.rados.0.trial006.stdout:41: writing trial00642507-41 from 672280 to 1464005 tid 1 2026-02-25T08:40:48.564 INFO:tasks.rados.rados.0.trial006.stdout:41: writing trial00642507-41 from 2157515 to 2672280 tid 2 2026-02-25T08:40:48.564 INFO:tasks.rados.rados.0.trial006.stdout:42: write initial oid 42 2026-02-25T08:40:48.565 INFO:tasks.rados.rados.0.trial006.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2026-02-25T08:40:48.569 INFO:tasks.rados.rados.0.trial006.stdout:42: writing trial00642507-42 from 689087 to 1156061 tid 1 2026-02-25T08:40:48.574 INFO:tasks.rados.rados.0.trial006.stdout:42: writing trial00642507-42 from 1699644 to 2483976 tid 2 2026-02-25T08:40:48.574 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:36: finishing write tid 2 to trial00642507-36 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:34: finishing write tid 4 to trial00642507-34 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:34: oid 34 updating version 0 to 10 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:34: oid 34 updating version 10 to 12 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:34: oid 34 version 12 is already newer than 12 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 34 v 12 (ObjNum 33 snap 0 seq_num 33) dirty exists 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:36: finishing write tid 3 to trial00642507-36 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:36: finishing write tid 4 to trial00642507-36 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:36: oid 36 updating version 0 to 6 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:36: oid 36 version 6 is already newer than 6 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:36: oid 36 version 6 is already newer than 4 2026-02-25T08:40:48.575 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2026-02-25T08:40:48.576 INFO:tasks.rados.rados.0.trial006.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2026-02-25T08:40:48.576 INFO:tasks.rados.rados.0.trial006.stdout:37: finishing write tid 2 to trial00642507-37 2026-02-25T08:40:48.576 INFO:tasks.rados.rados.0.trial006.stdout:34: done (15 left) 2026-02-25T08:40:48.576 INFO:tasks.rados.rados.0.trial006.stdout:36: done (14 left) 2026-02-25T08:40:48.576 INFO:tasks.rados.rados.0.trial006.stdout:43: write initial oid 43 2026-02-25T08:40:48.576 INFO:tasks.rados.rados.0.trial006.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2026-02-25T08:40:48.579 INFO:tasks.rados.rados.0.trial006.stdout:43: writing trial00642507-43 from 705894 to 1248117 tid 1 2026-02-25T08:40:48.583 INFO:tasks.rados.rados.0.trial006.stdout:43: writing trial00642507-43 from 1758126 to 2402469 tid 2 2026-02-25T08:40:48.583 INFO:tasks.rados.rados.0.trial006.stdout:44: write initial oid 44 2026-02-25T08:40:48.583 INFO:tasks.rados.rados.0.trial006.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2026-02-25T08:40:48.588 INFO:tasks.rados.rados.0.trial006.stdout:44: writing trial00642507-44 from 722701 to 1340173 tid 1 2026-02-25T08:40:48.594 INFO:tasks.rados.rados.0.trial006.stdout:44: writing trial00642507-44 from 1816608 to 2604609 tid 2 2026-02-25T08:40:48.594 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.594 INFO:tasks.rados.rados.0.trial006.stdout:27: finishing write tid 1 to trial00642507-27 2026-02-25T08:40:48.594 INFO:tasks.rados.rados.0.trial006.stdout:33: finishing write tid 2 to trial00642507-33 2026-02-25T08:40:48.594 INFO:tasks.rados.rados.0.trial006.stdout:39: finishing write tid 1 to trial00642507-39 2026-02-25T08:40:48.594 INFO:tasks.rados.rados.0.trial006.stdout:37: finishing write tid 3 to trial00642507-37 2026-02-25T08:40:48.594 INFO:tasks.rados.rados.0.trial006.stdout:37: finishing write tid 4 to trial00642507-37 2026-02-25T08:40:48.594 INFO:tasks.rados.rados.0.trial006.stdout:37: oid 37 updating version 0 to 6 2026-02-25T08:40:48.594 INFO:tasks.rados.rados.0.trial006.stdout:37: oid 37 version 6 is already newer than 4 2026-02-25T08:40:48.595 INFO:tasks.rados.rados.0.trial006.stdout:37: oid 37 version 6 is already newer than 6 2026-02-25T08:40:48.595 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2026-02-25T08:40:48.595 INFO:tasks.rados.rados.0.trial006.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2026-02-25T08:40:48.595 INFO:tasks.rados.rados.0.trial006.stdout:26: finishing write tid 1 to trial00642507-26 2026-02-25T08:40:48.595 INFO:tasks.rados.rados.0.trial006.stdout:37: done (15 left) 2026-02-25T08:40:48.595 INFO:tasks.rados.rados.0.trial006.stdout:45: write initial oid 45 2026-02-25T08:40:48.595 INFO:tasks.rados.rados.0.trial006.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2026-02-25T08:40:48.599 INFO:tasks.rados.rados.0.trial006.stdout:45: writing trial00642507-45 from 739508 to 1432229 tid 1 2026-02-25T08:40:48.604 INFO:tasks.rados.rados.0.trial006.stdout:45: writing trial00642507-45 from 1875090 to 2523102 tid 2 2026-02-25T08:40:48.604 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.604 INFO:tasks.rados.rados.0.trial006.stdout:33: finishing write tid 3 to trial00642507-33 2026-02-25T08:40:48.604 INFO:tasks.rados.rados.0.trial006.stdout:38: finishing write tid 1 to trial00642507-38 2026-02-25T08:40:48.604 INFO:tasks.rados.rados.0.trial006.stdout:33: finishing write tid 4 to trial00642507-33 2026-02-25T08:40:48.604 INFO:tasks.rados.rados.0.trial006.stdout:33: oid 33 updating version 0 to 4 2026-02-25T08:40:48.604 INFO:tasks.rados.rados.0.trial006.stdout:33: oid 33 updating version 4 to 6 2026-02-25T08:40:48.604 INFO:tasks.rados.rados.0.trial006.stdout:33: oid 33 version 6 is already newer than 6 2026-02-25T08:40:48.604 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 33 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2026-02-25T08:40:48.604 INFO:tasks.rados.rados.0.trial006.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2026-02-25T08:40:48.604 INFO:tasks.rados.rados.0.trial006.stdout:27: finishing write tid 2 to trial00642507-27 2026-02-25T08:40:48.605 INFO:tasks.rados.rados.0.trial006.stdout:33: done (15 left) 2026-02-25T08:40:48.605 INFO:tasks.rados.rados.0.trial006.stdout:46: write initial oid 46 2026-02-25T08:40:48.605 INFO:tasks.rados.rados.0.trial006.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2026-02-25T08:40:48.609 INFO:tasks.rados.rados.0.trial006.stdout:46: writing trial00642507-46 from 756315 to 1524285 tid 1 2026-02-25T08:40:48.614 INFO:tasks.rados.rados.0.trial006.stdout:46: writing trial00642507-46 from 1933572 to 2725242 tid 2 2026-02-25T08:40:48.614 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.614 INFO:tasks.rados.rados.0.trial006.stdout:40: finishing write tid 1 to trial00642507-40 2026-02-25T08:40:48.614 INFO:tasks.rados.rados.0.trial006.stdout:39: finishing write tid 2 to trial00642507-39 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:38: finishing write tid 2 to trial00642507-38 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:27: finishing write tid 3 to trial00642507-27 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:41: finishing write tid 1 to trial00642507-41 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:38: finishing write tid 3 to trial00642507-38 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:27: finishing write tid 4 to trial00642507-27 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:27: oid 27 updating version 0 to 10 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:27: oid 27 updating version 10 to 12 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:27: oid 27 version 12 is already newer than 12 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 27 v 12 (ObjNum 26 snap 0 seq_num 26) dirty exists 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:27: done (15 left) 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:47: write initial oid 47 2026-02-25T08:40:48.615 INFO:tasks.rados.rados.0.trial006.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2026-02-25T08:40:48.619 INFO:tasks.rados.rados.0.trial006.stdout:47: writing trial00642507-47 from 773122 to 1332694 tid 1 2026-02-25T08:40:48.624 INFO:tasks.rados.rados.0.trial006.stdout:47: writing trial00642507-47 from 1992054 to 2643735 tid 2 2026-02-25T08:40:48.624 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:39: finishing write tid 3 to trial00642507-39 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:26: finishing write tid 2 to trial00642507-26 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:43: finishing write tid 1 to trial00642507-43 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:39: finishing write tid 4 to trial00642507-39 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:39: oid 39 updating version 0 to 7 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:39: oid 39 updating version 7 to 9 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:39: oid 39 version 9 is already newer than 9 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 39 v 9 (ObjNum 38 snap 0 seq_num 38) dirty exists 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:39: done (15 left) 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:48: write initial oid 48 2026-02-25T08:40:48.625 INFO:tasks.rados.rados.0.trial006.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2026-02-25T08:40:48.629 INFO:tasks.rados.rados.0.trial006.stdout:48: writing trial00642507-48 from 789929 to 1424750 tid 1 2026-02-25T08:40:48.635 INFO:tasks.rados.rados.0.trial006.stdout:48: writing trial00642507-48 from 2050536 to 2789929 tid 2 2026-02-25T08:40:48.635 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.635 INFO:tasks.rados.rados.0.trial006.stdout:40: finishing write tid 2 to trial00642507-40 2026-02-25T08:40:48.635 INFO:tasks.rados.rados.0.trial006.stdout:38: finishing write tid 4 to trial00642507-38 2026-02-25T08:40:48.635 INFO:tasks.rados.rados.0.trial006.stdout:38: oid 38 updating version 0 to 9 2026-02-25T08:40:48.636 INFO:tasks.rados.rados.0.trial006.stdout:38: oid 38 version 9 is already newer than 7 2026-02-25T08:40:48.636 INFO:tasks.rados.rados.0.trial006.stdout:38: oid 38 version 9 is already newer than 9 2026-02-25T08:40:48.636 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 38 v 9 (ObjNum 37 snap 0 seq_num 37) dirty exists 2026-02-25T08:40:48.636 INFO:tasks.rados.rados.0.trial006.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2026-02-25T08:40:48.636 INFO:tasks.rados.rados.0.trial006.stdout:42: finishing write tid 1 to trial00642507-42 2026-02-25T08:40:48.636 INFO:tasks.rados.rados.0.trial006.stdout:38: done (15 left) 2026-02-25T08:40:48.636 INFO:tasks.rados.rados.0.trial006.stdout:49: write initial oid 49 2026-02-25T08:40:48.636 INFO:tasks.rados.rados.0.trial006.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2026-02-25T08:40:48.639 INFO:tasks.rados.rados.0.trial006.stdout:49: writing trial00642507-49 from 406736 to 1116806 tid 1 2026-02-25T08:40:48.645 INFO:tasks.rados.rados.0.trial006.stdout:49: writing trial00642507-49 from 1709018 to 2364368 tid 2 2026-02-25T08:40:48.645 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.645 INFO:tasks.rados.rados.0.trial006.stdout:26: finishing write tid 3 to trial00642507-26 2026-02-25T08:40:48.645 INFO:tasks.rados.rados.0.trial006.stdout:40: finishing write tid 3 to trial00642507-40 2026-02-25T08:40:48.645 INFO:tasks.rados.rados.0.trial006.stdout:26: finishing write tid 4 to trial00642507-26 2026-02-25T08:40:48.645 INFO:tasks.rados.rados.0.trial006.stdout:26: oid 26 updating version 0 to 10 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:26: oid 26 updating version 10 to 12 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:26: oid 26 version 12 is already newer than 12 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 26 v 12 (ObjNum 25 snap 0 seq_num 25) dirty exists 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:40: finishing write tid 4 to trial00642507-40 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:40: oid 40 updating version 0 to 9 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:40: oid 40 version 9 is already newer than 9 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:40: oid 40 version 9 is already newer than 7 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 40 v 9 (ObjNum 39 snap 0 seq_num 39) dirty exists 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:26: done (15 left) 2026-02-25T08:40:48.646 INFO:tasks.rados.rados.0.trial006.stdout:40: done (14 left) 2026-02-25T08:40:48.647 INFO:tasks.rados.rados.0.trial006.stdout:50: write initial oid 50 2026-02-25T08:40:48.647 INFO:tasks.rados.rados.0.trial006.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2026-02-25T08:40:48.649 INFO:tasks.rados.rados.0.trial006.stdout:50: writing trial00642507-50 from 423543 to 1208862 tid 1 2026-02-25T08:40:48.655 INFO:tasks.rados.rados.0.trial006.stdout:50: writing trial00642507-50 from 1767500 to 2566508 tid 2 2026-02-25T08:40:48.655 INFO:tasks.rados.rados.0.trial006.stdout:51: write (excl) oid 21 current snap is 0 2026-02-25T08:40:48.655 INFO:tasks.rados.rados.0.trial006.stdout:41: finishing write tid 2 to trial00642507-41 2026-02-25T08:40:48.655 INFO:tasks.rados.rados.0.trial006.stdout:51: seq_num 50 ranges {440350=460568,1309629=542666,2323973=516377} 2026-02-25T08:40:48.658 INFO:tasks.rados.rados.0.trial006.stdout:51: writing trial00642507-21 from 440350 to 900918 tid 1 2026-02-25T08:40:48.662 INFO:tasks.rados.rados.0.trial006.stdout:51: writing trial00642507-21 from 1309629 to 1852295 tid 2 2026-02-25T08:40:48.666 INFO:tasks.rados.rados.0.trial006.stdout:51: writing trial00642507-21 from 2323973 to 2840350 tid 3 2026-02-25T08:40:48.666 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.666 INFO:tasks.rados.rados.0.trial006.stdout:32: finishing write tid 1 to trial00642507-32 2026-02-25T08:40:48.666 INFO:tasks.rados.rados.0.trial006.stdout:41: finishing write tid 3 to trial00642507-41 2026-02-25T08:40:48.666 INFO:tasks.rados.rados.0.trial006.stdout:44: finishing write tid 1 to trial00642507-44 2026-02-25T08:40:48.667 INFO:tasks.rados.rados.0.trial006.stdout:41: finishing write tid 4 to trial00642507-41 2026-02-25T08:40:48.667 INFO:tasks.rados.rados.0.trial006.stdout:41: oid 41 updating version 0 to 6 2026-02-25T08:40:48.667 INFO:tasks.rados.rados.0.trial006.stdout:41: oid 41 version 6 is already newer than 6 2026-02-25T08:40:48.667 INFO:tasks.rados.rados.0.trial006.stdout:41: oid 41 version 6 is already newer than 4 2026-02-25T08:40:48.667 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 41 v 6 (ObjNum 40 snap 0 seq_num 40) dirty exists 2026-02-25T08:40:48.667 INFO:tasks.rados.rados.0.trial006.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2026-02-25T08:40:48.667 INFO:tasks.rados.rados.0.trial006.stdout:43: finishing write tid 2 to trial00642507-43 2026-02-25T08:40:48.667 INFO:tasks.rados.rados.0.trial006.stdout:41: done (15 left) 2026-02-25T08:40:48.667 INFO:tasks.rados.rados.0.trial006.stdout:52: write (excl) oid 14 current snap is 0 2026-02-25T08:40:48.667 INFO:tasks.rados.rados.0.trial006.stdout:46: finishing write tid 1 to trial00642507-46 2026-02-25T08:40:48.667 INFO:tasks.rados.rados.0.trial006.stdout:52: seq_num 51 ranges {457157=535817,1768111=402677} 2026-02-25T08:40:48.669 INFO:tasks.rados.rados.0.trial006.stdout:52: writing trial00642507-14 from 457157 to 992974 tid 1 2026-02-25T08:40:48.676 INFO:tasks.rados.rados.0.trial006.stdout:52: writing trial00642507-14 from 1768111 to 2170788 tid 2 2026-02-25T08:40:48.676 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.676 INFO:tasks.rados.rados.0.trial006.stdout:43: finishing write tid 3 to trial00642507-43 2026-02-25T08:40:48.676 INFO:tasks.rados.rados.0.trial006.stdout:43: finishing write tid 4 to trial00642507-43 2026-02-25T08:40:48.676 INFO:tasks.rados.rados.0.trial006.stdout:43: oid 43 updating version 0 to 12 2026-02-25T08:40:48.676 INFO:tasks.rados.rados.0.trial006.stdout:43: oid 43 version 12 is already newer than 10 2026-02-25T08:40:48.676 INFO:tasks.rados.rados.0.trial006.stdout:43: oid 43 version 12 is already newer than 12 2026-02-25T08:40:48.676 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 43 v 12 (ObjNum 42 snap 0 seq_num 42) dirty exists 2026-02-25T08:40:48.676 INFO:tasks.rados.rados.0.trial006.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:45: finishing write tid 1 to trial00642507-45 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:42: finishing write tid 2 to trial00642507-42 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:32: finishing write tid 2 to trial00642507-32 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:15: finishing write tid 1 to trial00642507-15 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:44: finishing write tid 2 to trial00642507-44 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:49: finishing write tid 1 to trial00642507-49 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:43: done (15 left) 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:53: read oid 22 snap -1 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:53: expect (ObjNum 21 snap 0 seq_num 21) 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:44: finishing write tid 3 to trial00642507-44 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:44: finishing write tid 4 to trial00642507-44 2026-02-25T08:40:48.677 INFO:tasks.rados.rados.0.trial006.stdout:44: oid 44 updating version 0 to 12 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout:44: oid 44 version 12 is already newer than 12 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout:44: oid 44 version 12 is already newer than 10 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 44 v 12 (ObjNum 43 snap 0 seq_num 43) dirty exists 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout:48: finishing write tid 1 to trial00642507-48 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout:47: finishing write tid 1 to trial00642507-47 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout:45: finishing write tid 2 to trial00642507-45 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout:32: finishing write tid 3 to trial00642507-32 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout:44: done (15 left) 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout:54: read oid 36 snap -1 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout:54: expect (ObjNum 35 snap 0 seq_num 35) 2026-02-25T08:40:48.678 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.679 INFO:tasks.rados.rados.0.trial006.stdout:42: finishing write tid 3 to trial00642507-42 2026-02-25T08:40:48.680 INFO:tasks.rados.rados.0.trial006.stdout:45: finishing write tid 3 to trial00642507-45 2026-02-25T08:40:48.681 INFO:tasks.rados.rados.0.trial006.stdout:32: finishing write tid 4 to trial00642507-32 2026-02-25T08:40:48.681 INFO:tasks.rados.rados.0.trial006.stdout:32: oid 32 updating version 0 to 13 2026-02-25T08:40:48.681 INFO:tasks.rados.rados.0.trial006.stdout:32: oid 32 updating version 13 to 15 2026-02-25T08:40:48.681 INFO:tasks.rados.rados.0.trial006.stdout:32: oid 32 version 15 is already newer than 15 2026-02-25T08:40:48.681 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 32 v 15 (ObjNum 31 snap 0 seq_num 31) dirty exists 2026-02-25T08:40:48.681 INFO:tasks.rados.rados.0.trial006.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2026-02-25T08:40:48.681 INFO:tasks.rados.rados.0.trial006.stdout:32: done (15 left) 2026-02-25T08:40:48.681 INFO:tasks.rados.rados.0.trial006.stdout:55: read oid 27 snap -1 2026-02-25T08:40:48.681 INFO:tasks.rados.rados.0.trial006.stdout:55: expect (ObjNum 26 snap 0 seq_num 26) 2026-02-25T08:40:48.682 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.682 INFO:tasks.rados.rados.0.trial006.stdout:45: finishing write tid 4 to trial00642507-45 2026-02-25T08:40:48.682 INFO:tasks.rados.rados.0.trial006.stdout:45: oid 45 updating version 0 to 15 2026-02-25T08:40:48.683 INFO:tasks.rados.rados.0.trial006.stdout:45: oid 45 version 15 is already newer than 15 2026-02-25T08:40:48.683 INFO:tasks.rados.rados.0.trial006.stdout:45: oid 45 version 15 is already newer than 13 2026-02-25T08:40:48.683 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 45 v 15 (ObjNum 44 snap 0 seq_num 44) dirty exists 2026-02-25T08:40:48.683 INFO:tasks.rados.rados.0.trial006.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2026-02-25T08:40:48.683 INFO:tasks.rados.rados.0.trial006.stdout:45: done (15 left) 2026-02-25T08:40:48.683 INFO:tasks.rados.rados.0.trial006.stdout:56: read oid 31 snap -1 2026-02-25T08:40:48.683 INFO:tasks.rados.rados.0.trial006.stdout:56: expect (ObjNum 30 snap 0 seq_num 30) 2026-02-25T08:40:48.683 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.684 INFO:tasks.rados.rados.0.trial006.stdout:42: finishing write tid 4 to trial00642507-42 2026-02-25T08:40:48.684 INFO:tasks.rados.rados.0.trial006.stdout:42: oid 42 updating version 0 to 3 2026-02-25T08:40:48.684 INFO:tasks.rados.rados.0.trial006.stdout:42: oid 42 version 3 is already newer than 1 2026-02-25T08:40:48.684 INFO:tasks.rados.rados.0.trial006.stdout:42: oid 42 version 3 is already newer than 3 2026-02-25T08:40:48.684 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 42 v 3 (ObjNum 41 snap 0 seq_num 41) dirty exists 2026-02-25T08:40:48.684 INFO:tasks.rados.rados.0.trial006.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2026-02-25T08:40:48.684 INFO:tasks.rados.rados.0.trial006.stdout:42: done (15 left) 2026-02-25T08:40:48.685 INFO:tasks.rados.rados.0.trial006.stdout:57: read oid 30 snap -1 2026-02-25T08:40:48.685 INFO:tasks.rados.rados.0.trial006.stdout:57: expect (ObjNum 29 snap 0 seq_num 29) 2026-02-25T08:40:48.685 INFO:tasks.rados.rados.0.trial006.stdout: waiting on 16 2026-02-25T08:40:48.685 INFO:tasks.rados.rados.0.trial006.stdout:49: finishing write tid 2 to trial00642507-49 2026-02-25T08:40:48.686 INFO:tasks.rados.rados.0.trial006.stdout:46: finishing write tid 2 to trial00642507-46 2026-02-25T08:40:48.691 INFO:tasks.rados.rados.0.trial006.stdout:15: finishing write tid 2 to trial00642507-15 2026-02-25T08:40:48.691 INFO:tasks.rados.rados.0.trial006.stdout:47: finishing write tid 2 to trial00642507-47 2026-02-25T08:40:48.695 INFO:tasks.rados.rados.0.trial006.stdout:15: finishing write tid 3 to trial00642507-15 2026-02-25T08:40:48.695 INFO:tasks.rados.rados.0.trial006.stdout:46: finishing write tid 3 to trial00642507-46 2026-02-25T08:40:48.697 INFO:tasks.rados.rados.0.trial006.stdout:46: finishing write tid 4 to trial00642507-46 2026-02-25T08:40:48.698 INFO:tasks.rados.rados.0.trial006.stdout:46: oid 46 updating version 0 to 10 2026-02-25T08:40:48.698 INFO:tasks.rados.rados.0.trial006.stdout:46: oid 46 updating version 10 to 12 2026-02-25T08:40:48.698 INFO:tasks.rados.rados.0.trial006.stdout:46: oid 46 version 12 is already newer than 12 2026-02-25T08:40:48.698 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 46 v 12 (ObjNum 45 snap 0 seq_num 45) dirty exists 2026-02-25T08:40:48.698 INFO:tasks.rados.rados.0.trial006.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2026-02-25T08:40:48.698 INFO:tasks.rados.rados.0.trial006.stdout:46: done (15 left) 2026-02-25T08:40:48.698 INFO:tasks.rados.rados.0.trial006.stdout:58: delete oid 6 current snap is 0 2026-02-25T08:40:48.698 INFO:tasks.rados.rados.0.trial006.stdout:47: finishing write tid 3 to trial00642507-47 2026-02-25T08:40:48.699 INFO:tasks.rados.rados.0.trial006.stdout:47: finishing write tid 4 to trial00642507-47 2026-02-25T08:40:48.699 INFO:tasks.rados.rados.0.trial006.stdout:47: oid 47 updating version 0 to 15 2026-02-25T08:40:48.699 INFO:tasks.rados.rados.0.trial006.stdout:47: oid 47 version 15 is already newer than 15 2026-02-25T08:40:48.699 INFO:tasks.rados.rados.0.trial006.stdout:47: oid 47 version 15 is already newer than 13 2026-02-25T08:40:48.699 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 47 v 15 (ObjNum 46 snap 0 seq_num 46) dirty exists 2026-02-25T08:40:48.700 INFO:tasks.rados.rados.0.trial006.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2026-02-25T08:40:48.700 INFO:tasks.rados.rados.0.trial006.stdout:50: finishing write tid 1 to trial00642507-50 2026-02-25T08:40:48.701 INFO:tasks.rados.rados.0.trial006.stdout:15: finishing write tid 4 to trial00642507-15 2026-02-25T08:40:48.701 INFO:tasks.rados.rados.0.trial006.stdout:15: oid 15 updating version 0 to 3 2026-02-25T08:40:48.701 INFO:tasks.rados.rados.0.trial006.stdout:15: oid 15 version 3 is already newer than 1 2026-02-25T08:40:48.701 INFO:tasks.rados.rados.0.trial006.stdout:15: oid 15 version 3 is already newer than 3 2026-02-25T08:40:48.701 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 15 v 3 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-25T08:40:48.701 INFO:tasks.rados.rados.0.trial006.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2026-02-25T08:40:48.701 INFO:tasks.rados.rados.0.trial006.stdout:51: finishing write tid 1 to trial00642507-21 2026-02-25T08:40:48.702 INFO:tasks.rados.rados.0.trial006.stdout:19: finishing write tid 1 to trial00642507-19 2026-02-25T08:40:48.707 INFO:tasks.rados.rados.0.trial006.stdout:49: finishing write tid 3 to trial00642507-49 2026-02-25T08:40:48.710 INFO:tasks.rados.rados.0.trial006.stdout:49: finishing write tid 4 to trial00642507-49 2026-02-25T08:40:48.710 INFO:tasks.rados.rados.0.trial006.stdout:49: oid 49 updating version 0 to 6 2026-02-25T08:40:48.710 INFO:tasks.rados.rados.0.trial006.stdout:49: oid 49 version 6 is already newer than 6 2026-02-25T08:40:48.710 INFO:tasks.rados.rados.0.trial006.stdout:49: oid 49 version 6 is already newer than 4 2026-02-25T08:40:48.710 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 49 v 6 (ObjNum 48 snap 0 seq_num 48) dirty exists 2026-02-25T08:40:48.710 INFO:tasks.rados.rados.0.trial006.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2026-02-25T08:40:48.714 INFO:tasks.rados.rados.0.trial006.stdout:48: finishing write tid 2 to trial00642507-48 2026-02-25T08:40:48.719 INFO:tasks.rados.rados.0.trial006.stdout:57: expect (ObjNum 29 snap 0 seq_num 29) 2026-02-25T08:40:48.729 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 0 2026-02-25T08:40:48.772 INFO:tasks.rados.rados.0.trial006.stdout:15: done (15 left) 2026-02-25T08:40:48.773 INFO:tasks.rados.rados.0.trial006.stdout:47: done (14 left) 2026-02-25T08:40:48.773 INFO:tasks.rados.rados.0.trial006.stdout:49: done (13 left) 2026-02-25T08:40:48.773 INFO:tasks.rados.rados.0.trial006.stdout:57: done (12 left) 2026-02-25T08:40:48.773 INFO:tasks.rados.rados.0.trial006.stdout:58: done (11 left) 2026-02-25T08:40:48.773 INFO:tasks.rados.rados.0.trial006.stdout:59: write oid 39 current snap is 0 2026-02-25T08:40:48.773 INFO:tasks.rados.rados.0.trial006.stdout:48: finishing write tid 3 to trial00642507-48 2026-02-25T08:40:48.773 INFO:tasks.rados.rados.0.trial006.stdout:48: finishing write tid 4 to trial00642507-48 2026-02-25T08:40:48.773 INFO:tasks.rados.rados.0.trial006.stdout:48: oid 48 updating version 0 to 9 2026-02-25T08:40:48.773 INFO:tasks.rados.rados.0.trial006.stdout:48: oid 48 version 9 is already newer than 7 2026-02-25T08:40:48.773 INFO:tasks.rados.rados.0.trial006.stdout:48: oid 48 version 9 is already newer than 9 2026-02-25T08:40:48.773 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 48 v 9 (ObjNum 47 snap 0 seq_num 47) dirty exists 2026-02-25T08:40:48.774 INFO:tasks.rados.rados.0.trial006.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2026-02-25T08:40:48.774 INFO:tasks.rados.rados.0.trial006.stdout:55: expect (ObjNum 26 snap 0 seq_num 26) 2026-02-25T08:40:48.790 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 2107738 2026-02-25T08:40:48.792 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 0 2026-02-25T08:40:48.813 INFO:tasks.rados.rados.0.trial006.stdout:19: finishing write tid 2 to trial00642507-19 2026-02-25T08:40:48.813 INFO:tasks.rados.rados.0.trial006.stdout:56: expect (ObjNum 30 snap 0 seq_num 30) 2026-02-25T08:40:48.815 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 598773 2026-02-25T08:40:48.836 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 2394964 2026-02-25T08:40:48.861 INFO:tasks.rados.rados.0.trial006.stdout:54: expect (ObjNum 35 snap 0 seq_num 35) 2026-02-25T08:40:48.879 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 1358123 2026-02-25T08:40:48.902 INFO:tasks.rados.rados.0.trial006.stdout:19: finishing write tid 3 to trial00642507-19 2026-02-25T08:40:48.903 INFO:tasks.rados.rados.0.trial006.stdout:59: seq_num 53 ranges {490771=686315,1885075=406346} 2026-02-25T08:40:48.907 INFO:tasks.rados.rados.0.trial006.stdout:59: writing trial00642507-39 from 490771 to 1177086 tid 1 2026-02-25T08:40:48.911 INFO:tasks.rados.rados.0.trial006.stdout:59: writing trial00642507-39 from 1885075 to 2291421 tid 2 2026-02-25T08:40:48.912 INFO:tasks.rados.rados.0.trial006.stdout:48: done (11 left) 2026-02-25T08:40:48.912 INFO:tasks.rados.rados.0.trial006.stdout:54: done (10 left) 2026-02-25T08:40:48.912 INFO:tasks.rados.rados.0.trial006.stdout:55: done (9 left) 2026-02-25T08:40:48.912 INFO:tasks.rados.rados.0.trial006.stdout:56: done (8 left) 2026-02-25T08:40:48.912 INFO:tasks.rados.rados.0.trial006.stdout:60: write oid 15 current snap is 0 2026-02-25T08:40:48.912 INFO:tasks.rados.rados.0.trial006.stdout:60: seq_num 54 ranges {507578=477917,1543557=550004,2833665=73913} 2026-02-25T08:40:48.915 INFO:tasks.rados.rados.0.trial006.stdout:60: writing trial00642507-15 from 507578 to 985495 tid 1 2026-02-25T08:40:48.919 INFO:tasks.rados.rados.0.trial006.stdout:60: writing trial00642507-15 from 1543557 to 2093561 tid 2 2026-02-25T08:40:48.924 INFO:tasks.rados.rados.0.trial006.stdout:60: writing trial00642507-15 from 2833665 to 2907578 tid 3 2026-02-25T08:40:48.924 INFO:tasks.rados.rados.0.trial006.stdout:61: read oid 18 snap -1 2026-02-25T08:40:48.924 INFO:tasks.rados.rados.0.trial006.stdout:61: expect (ObjNum 17 snap 0 seq_num 17) 2026-02-25T08:40:48.924 INFO:tasks.rados.rados.0.trial006.stdout:62: snap_create 2026-02-25T08:40:48.924 INFO:tasks.rados.rados.0.trial006.stdout:19: finishing write tid 4 to trial00642507-19 2026-02-25T08:40:48.924 INFO:tasks.rados.rados.0.trial006.stdout:19: oid 19 updating version 0 to 6 2026-02-25T08:40:48.924 INFO:tasks.rados.rados.0.trial006.stdout:19: oid 19 version 6 is already newer than 6 2026-02-25T08:40:48.924 INFO:tasks.rados.rados.0.trial006.stdout:19: oid 19 version 6 is already newer than 4 2026-02-25T08:40:48.924 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 19 v 6 (ObjNum 18 snap 0 seq_num 18) dirty exists 2026-02-25T08:40:48.925 INFO:tasks.rados.rados.0.trial006.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2026-02-25T08:40:48.925 INFO:tasks.rados.rados.0.trial006.stdout:51: finishing write tid 2 to trial00642507-21 2026-02-25T08:40:48.925 INFO:tasks.rados.rados.0.trial006.stdout:50: finishing write tid 2 to trial00642507-50 2026-02-25T08:40:48.925 INFO:tasks.rados.rados.0.trial006.stdout:50: finishing write tid 3 to trial00642507-50 2026-02-25T08:40:48.925 INFO:tasks.rados.rados.0.trial006.stdout:50: finishing write tid 4 to trial00642507-50 2026-02-25T08:40:48.925 INFO:tasks.rados.rados.0.trial006.stdout:50: oid 50 updating version 0 to 18 2026-02-25T08:40:48.925 INFO:tasks.rados.rados.0.trial006.stdout:50: oid 50 version 18 is already newer than 18 2026-02-25T08:40:48.925 INFO:tasks.rados.rados.0.trial006.stdout:50: oid 50 version 18 is already newer than 16 2026-02-25T08:40:48.925 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 50 v 18 (ObjNum 49 snap 0 seq_num 49) dirty exists 2026-02-25T08:40:48.925 INFO:tasks.rados.rados.0.trial006.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2026-02-25T08:40:48.926 INFO:tasks.rados.rados.0.trial006.stdout:25: finishing write tid 1 to trial00642507-25 2026-02-25T08:40:48.926 INFO:tasks.rados.rados.0.trial006.stdout:52: finishing write tid 1 to trial00642507-14 2026-02-25T08:40:48.926 INFO:tasks.rados.rados.0.trial006.stdout:25: finishing write tid 2 to trial00642507-25 2026-02-25T08:40:48.926 INFO:tasks.rados.rados.0.trial006.stdout:51: finishing write tid 3 to trial00642507-21 2026-02-25T08:40:48.926 INFO:tasks.rados.rados.0.trial006.stdout:51: finishing write tid 4 to trial00642507-21 2026-02-25T08:40:48.926 INFO:tasks.rados.rados.0.trial006.stdout:51: finishing write tid 5 to trial00642507-21 2026-02-25T08:40:48.926 INFO:tasks.rados.rados.0.trial006.stdout:51: oid 21 updating version 0 to 4 2026-02-25T08:40:48.926 INFO:tasks.rados.rados.0.trial006.stdout:51: oid 21 updating version 4 to 7 2026-02-25T08:40:48.926 INFO:tasks.rados.rados.0.trial006.stdout:51: oid 21 version 7 is already newer than 7 2026-02-25T08:40:48.926 INFO:tasks.rados.rados.0.trial006.stdout:51: oid 21 version 7 is already newer than 7 2026-02-25T08:40:48.926 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 21 v 7 (ObjNum 50 snap 0 seq_num 50) dirty exists 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:51: left oid 21 (ObjNum 50 snap 0 seq_num 50) 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:25: finishing write tid 3 to trial00642507-25 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:25: finishing write tid 4 to trial00642507-25 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:25: oid 25 updating version 0 to 7 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:25: oid 25 updating version 7 to 9 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:25: oid 25 version 9 is already newer than 9 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 25 v 9 (ObjNum 24 snap 0 seq_num 24) dirty exists 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:28: finishing write tid 1 to trial00642507-28 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:52: finishing write tid 2 to trial00642507-14 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:53: expect (ObjNum 21 snap 0 seq_num 21) 2026-02-25T08:40:48.927 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 0 2026-02-25T08:40:48.946 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 1670417 2026-02-25T08:40:48.954 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 0 2026-02-25T08:40:48.975 INFO:tasks.rados.rados.0.trial006.stdout:52: finishing write tid 3 to trial00642507-14 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:52: finishing write tid 4 to trial00642507-14 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:52: oid 14 updating version 0 to 18 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:52: oid 14 version 18 is already newer than 18 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:52: oid 14 version 18 is already newer than 16 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 14 v 18 (ObjNum 51 snap 0 seq_num 51) dirty exists 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:52: left oid 14 (ObjNum 51 snap 0 seq_num 51) 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:28: finishing write tid 2 to trial00642507-28 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:28: finishing write tid 3 to trial00642507-28 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:28: finishing write tid 4 to trial00642507-28 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:28: oid 28 updating version 0 to 12 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:28: oid 28 version 12 is already newer than 12 2026-02-25T08:40:48.976 INFO:tasks.rados.rados.0.trial006.stdout:28: oid 28 version 12 is already newer than 10 2026-02-25T08:40:48.977 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 28 v 12 (ObjNum 27 snap 0 seq_num 27) dirty exists 2026-02-25T08:40:48.977 INFO:tasks.rados.rados.0.trial006.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2026-02-25T08:40:48.977 INFO:tasks.rados.rados.0.trial006.stdout:59: finishing write tid 1 to trial00642507-39 2026-02-25T08:40:48.977 INFO:tasks.rados.rados.0.trial006.stdout:61: expect (ObjNum 17 snap 0 seq_num 17) 2026-02-25T08:40:49.017 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 1180408 2026-02-25T08:40:49.027 INFO:tasks.rados.rados.0.trial006.stdout:60: finishing write tid 1 to trial00642507-15 2026-02-25T08:40:49.027 INFO:tasks.rados.rados.0.trial006.stdout:59: finishing write tid 2 to trial00642507-39 2026-02-25T08:40:49.027 INFO:tasks.rados.rados.0.trial006.stdout:60: finishing write tid 2 to trial00642507-15 2026-02-25T08:40:49.027 INFO:tasks.rados.rados.0.trial006.stdout:59: finishing write tid 3 to trial00642507-39 2026-02-25T08:40:49.027 INFO:tasks.rados.rados.0.trial006.stdout:60: finishing write tid 3 to trial00642507-15 2026-02-25T08:40:49.027 INFO:tasks.rados.rados.0.trial006.stdout:59: finishing write tid 4 to trial00642507-39 2026-02-25T08:40:49.027 INFO:tasks.rados.rados.0.trial006.stdout:59: oid 39 updating version 0 to 15 2026-02-25T08:40:49.027 INFO:tasks.rados.rados.0.trial006.stdout:59: oid 39 version 15 is already newer than 15 2026-02-25T08:40:49.027 INFO:tasks.rados.rados.0.trial006.stdout:59: oid 39 version 15 is already newer than 15 2026-02-25T08:40:49.027 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 39 v 15 (ObjNum 53 snap 0 seq_num 53) dirty exists 2026-02-25T08:40:49.027 INFO:tasks.rados.rados.0.trial006.stdout:59: left oid 39 (ObjNum 53 snap 0 seq_num 53) 2026-02-25T08:40:49.028 INFO:tasks.rados.rados.0.trial006.stdout:60: finishing write tid 4 to trial00642507-15 2026-02-25T08:40:49.028 INFO:tasks.rados.rados.0.trial006.stdout:60: finishing write tid 5 to trial00642507-15 2026-02-25T08:40:49.028 INFO:tasks.rados.rados.0.trial006.stdout:60: oid 15 updating version 0 to 16 2026-02-25T08:40:49.028 INFO:tasks.rados.rados.0.trial006.stdout:60: oid 15 version 16 is already newer than 16 2026-02-25T08:40:49.028 INFO:tasks.rados.rados.0.trial006.stdout:60: oid 15 version 16 is already newer than 16 2026-02-25T08:40:49.028 INFO:tasks.rados.rados.0.trial006.stdout:60: oid 15 version 16 is already newer than 16 2026-02-25T08:40:49.028 INFO:tasks.rados.rados.0.trial006.stdout:update_object_version oid 15 v 16 (ObjNum 54 snap 0 seq_num 54) dirty exists 2026-02-25T08:40:49.028 INFO:tasks.rados.rados.0.trial006.stdout:60: left oid 15 (ObjNum 54 snap 0 seq_num 54) 2026-02-25T08:40:49.203 INFO:tasks.rados.rados.0.trial006.stdout:19: done (10 left) 2026-02-25T08:40:49.203 INFO:tasks.rados.rados.0.trial006.stdout:25: done (9 left) 2026-02-25T08:40:49.203 INFO:tasks.rados.rados.0.trial006.stdout:28: done (8 left) 2026-02-25T08:40:49.203 INFO:tasks.rados.rados.0.trial006.stdout:50: done (7 left) 2026-02-25T08:40:49.204 INFO:tasks.rados.rados.0.trial006.stdout:51: done (6 left) 2026-02-25T08:40:49.204 INFO:tasks.rados.rados.0.trial006.stdout:52: done (5 left) 2026-02-25T08:40:49.204 INFO:tasks.rados.rados.0.trial006.stdout:53: done (4 left) 2026-02-25T08:40:49.204 INFO:tasks.rados.rados.0.trial006.stdout:59: done (3 left) 2026-02-25T08:40:49.204 INFO:tasks.rados.rados.0.trial006.stdout:60: done (2 left) 2026-02-25T08:40:49.204 INFO:tasks.rados.rados.0.trial006.stdout:61: done (1 left) 2026-02-25T08:40:49.204 INFO:tasks.rados.rados.0.trial006.stdout:62: done (0 left) 2026-02-25T08:40:49.204 INFO:tasks.rados.rados.0.trial006.stdout:63: read oid 49 snap -1 2026-02-25T08:40:49.204 INFO:tasks.rados.rados.0.trial006.stdout:63: expect (ObjNum 48 snap 0 seq_num 48) 2026-02-25T08:40:49.204 INFO:tasks.rados.rados.0.trial006.stdout:64: delete oid 24 current snap is 1 2026-02-25T08:40:49.233 INFO:tasks.rados.rados.0.trial006.stdout:63: expect (ObjNum 48 snap 0 seq_num 48) 2026-02-25T08:40:49.304 INFO:tasks.rados.rados.0.trial006.stdout:63: done (1 left) 2026-02-25T08:40:49.304 INFO:tasks.rados.rados.0.trial006.stdout:64: done (0 left) 2026-02-25T08:40:49.305 INFO:tasks.rados.rados.0.trial006.stdout:65: read oid 39 snap 0 2026-02-25T08:40:49.305 INFO:tasks.rados.rados.0.trial006.stdout:65: expect (ObjNum 53 snap 0 seq_num 53) 2026-02-25T08:40:49.305 INFO:tasks.rados.rados.0.trial006.stdout:66: read oid 16 snap -1 2026-02-25T08:40:49.305 INFO:tasks.rados.rados.0.trial006.stdout:66: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-25T08:40:49.305 INFO:tasks.rados.rados.0.trial006.stdout:67: write oid 45 current snap is 1 2026-02-25T08:40:49.305 INFO:tasks.rados.rados.0.trial006.stdout:67: seq_num 56 ranges {541192=628415,1660521=553673,2888511=52681} 2026-02-25T08:40:49.308 INFO:tasks.rados.rados.0.trial006.stdout:67: writing trial00642507-45 from 541192 to 1169607 tid 1 2026-02-25T08:40:49.312 INFO:tasks.rados.rados.0.trial006.stdout:67: writing trial00642507-45 from 1660521 to 2214194 tid 2 2026-02-25T08:40:49.315 INFO:tasks.rados.rados.0.trial006.stdout:67: writing trial00642507-45 from 2888511 to 2941192 tid 3 2026-02-25T08:40:49.315 INFO:tasks.rados.rados.0.trial006.stdout:68: read oid 48 snap -1 2026-02-25T08:40:49.316 INFO:tasks.rados.rados.0.trial006.stdout:68: expect (ObjNum 47 snap 0 seq_num 47) 2026-02-25T08:40:49.316 INFO:tasks.rados.rados.0.trial006.stdout:69: snap_create 2026-02-25T08:40:49.318 INFO:tasks.rados.rados.0.trial006.stdout:66: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-25T08:40:49.318 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 0 2026-02-25T08:40:49.333 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-25T08:40:49.334 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2026-02-25T08:40:49.334 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 3] 2026-02-25T08:40:49.334 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-25T08:40:49.335 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:40:49.375 INFO:tasks.rados.rados.0.trial006.stdout:65: expect (ObjNum 53 snap 0 seq_num 53) 2026-02-25T08:40:49.375 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 0 2026-02-25T08:40:49.415 INFO:tasks.ceph.osd.0:Stopped 2026-02-25T08:40:49.431 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 0 2026-02-25T08:40:49.465 INFO:tasks.rados.rados.0.trial006.stdout:68: expect (ObjNum 47 snap 0 seq_num 47) 2026-02-25T08:40:49.466 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 0 2026-02-25T08:40:49.503 INFO:tasks.rados.rados.0.trial006.stdout:seeking to 995082 2026-02-25T08:40:49.517 INFO:tasks.rados.rados.0.trial006.stdout:67: finishing write tid 1 to trial00642507-45 2026-02-25T08:40:50.205 INFO:tasks.rados.rados.0.trial006.stdout:65: done (4 left) 2026-02-25T08:40:50.206 INFO:tasks.rados.rados.0.trial006.stdout:66: done (3 left) 2026-02-25T08:40:50.206 INFO:tasks.rados.rados.0.trial006.stdout:68: done (2 left) 2026-02-25T08:40:50.206 INFO:tasks.rados.rados.0.trial006.stdout:69: done (1 left) 2026-02-25T08:40:50.206 INFO:tasks.rados.rados.0.trial006.stdout:70: delete oid 12 current snap is 2 2026-02-25T08:40:50.653 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:40:50.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:50.728+0000 7f369f68b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:50.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:40:50.732+0000 7f369f68b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:54.415 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3] 2026-02-25T08:40:54.430 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2026-02-25T08:40:54.431 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [1, 2, 3] 2026-02-25T08:40:54.431 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-25T08:40:54.431 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:40:54.504 INFO:tasks.ceph.osd.1:Stopped 2026-02-25T08:40:59.505 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0, 1] live_osds: [2, 3] 2026-02-25T08:40:59.505 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2026-02-25T08:40:59.506 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2026-02-25T08:40:59.506 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-25T08:40:59.506 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-25T08:40:59.509 INFO:tasks.ceph.osd.1:Started 2026-02-25T08:40:59.509 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:40:59.583 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:40:59.585 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:40:59.586 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:04.587 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:04.658 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:04.661 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:04.662 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:09.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2026-02-25T08:41:09.663 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:09.736 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:09.738 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:09.739 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:14.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~5s 2026-02-25T08:41:14.740 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:14.813 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:14.816 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:14.816 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:18.819 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:19.818 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:19.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~11s 2026-02-25T08:41:19.890 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:19.893 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:19.894 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:24.895 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:24.968 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:24.970 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:24.971 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:25.289 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~16s 2026-02-25T08:41:25.742 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:25.770 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:25.771 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:27.772 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:41:27.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:27.846+0000 7ff2627ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:27.850+0000 7ff2627ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.855 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:27.855 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:27.887 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:27.887 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:29.888 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:41:29.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:29.962+0000 7fafcee5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:29.965+0000 7fafcee5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.971 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:29.971 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:29.972 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:29.996 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:29.996 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:30.041 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:30.044 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:30.045 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:30.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~22s 2026-02-25T08:41:31.997 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:41:32.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:32.071+0000 7f49a3e1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:32.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:32.075+0000 7f49a3e1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:32.080 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:32.080 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:32.107 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:32.108 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:34.109 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:41:34.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:34.184+0000 7f68de61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:34.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:34.187+0000 7f68de61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:34.193 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:34.193 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:34.219 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:34.219 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:35.045 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:35.117 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:35.120 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:35.121 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:36.096 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~27s 2026-02-25T08:41:36.220 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:41:36.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:36.294+0000 7f58c6c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:36.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:36.298+0000 7f58c6c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:36.304 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:36.304 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:36.330 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:36.330 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:38.332 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:41:38.410 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:38.408+0000 7f46f485a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:38.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:38.411+0000 7f46f485a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:38.416 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:38.416 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:38.443 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:38.444 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:40.123 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:40.195 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:40.197 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:40.198 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:40.445 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:41:40.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:40.520+0000 7ff7d1ffd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:40.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:40.524+0000 7ff7d1ffd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:40.528 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:40.528 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:40.555 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:40.556 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:41.500 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~32s 2026-02-25T08:41:42.557 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:41:42.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:42.633+0000 7f5e46fb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:42.637+0000 7f5e46fb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:42.641 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:42.641 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:42.666 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:42.666 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:44.666 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:41:44.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:44.741+0000 7f7cdd45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:44.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:44.745+0000 7f7cdd45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:44.749 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:44.749 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:44.776 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:44.776 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:45.199 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:45.271 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:45.274 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:45.274 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:46.776 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:41:46.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:46.851+0000 7fec22d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:46.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:46.855+0000 7fec22d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:46.860 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:46.860 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:46.889 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:46.890 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:46.904 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~38s 2026-02-25T08:41:48.891 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:41:48.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:48.967+0000 7f4213835640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:48.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:48.971+0000 7f4213835640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:48.975 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:48.975 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:49.002 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:49.003 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:50.275 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:50.348 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:50.351 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:50.352 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:51.004 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:41:51.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:51.079+0000 7f35403dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:51.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:51.082+0000 7f35403dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:51.087 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:51.087 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:51.113 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:51.114 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:52.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~43s 2026-02-25T08:41:53.115 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:41:53.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:53.191+0000 7f8e5dccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:53.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:53.195+0000 7f8e5dccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:53.199 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:53.199 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:53.225 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:53.225 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:55.227 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:41:55.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:55.303+0000 7f5f2c35b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:55.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:55.306+0000 7f5f2c35b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:55.311 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:55.311 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:55.336 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:55.336 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:55.353 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:55.423 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:55.426 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:55.426 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:57.337 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:41:57.415 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:57.413+0000 7f69b0774640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:57.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:57.417+0000 7f69b0774640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:57.421 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:57.421 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:57.448 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:57.449 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:57.710 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~49s 2026-02-25T08:41:59.450 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:41:59.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:59.526+0000 7f4538d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:59.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:41:59.530+0000 7f4538d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:59.534 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:41:59.534 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:59.559 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:59.559 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:00.428 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:00.501 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:00.504 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:00.505 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:01.561 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:01.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:01.636+0000 7ff739811640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:01.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:01.640+0000 7ff739811640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:01.645 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:01.645 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:01.670 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:01.670 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:03.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~54s 2026-02-25T08:42:03.672 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:03.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:03.748+0000 7f89a4e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:03.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:03.752+0000 7f89a4e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:03.756 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:03.757 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:03.784 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:03.784 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:05.506 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:05.580 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:05.583 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:05.584 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:05.785 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:05.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:05.858+0000 7fb257ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:05.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:05.861+0000 7fb257ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:05.867 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:05.867 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:05.892 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:05.892 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:07.893 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:07.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:07.968+0000 7f05e2e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:07.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:07.971+0000 7f05e2e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:07.977 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:07.977 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:08.001 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:08.002 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:08.516 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~59s 2026-02-25T08:42:10.003 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:10.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:10.078+0000 7f222765a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:10.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:10.081+0000 7f222765a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:10.087 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:10.087 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:10.115 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:10.115 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:10.584 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:10.656 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:10.659 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:10.659 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:12.116 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:12.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:12.190+0000 7fb02625a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:12.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:12.194+0000 7fb02625a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:12.199 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:12.199 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:12.226 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:12.226 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:13.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~65s 2026-02-25T08:42:14.227 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:14.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:14.301+0000 7f3e63b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:14.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:14.305+0000 7f3e63b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:14.311 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:14.311 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:14.336 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:14.336 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:15.660 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:15.733 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:15.736 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:15.736 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:16.337 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:16.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:16.411+0000 7f8c91c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:16.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:16.415+0000 7f8c91c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:16.420 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:16.420 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:16.447 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:16.448 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:18.449 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:18.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:18.526+0000 7f8db0f62640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:18.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:18.530+0000 7f8db0f62640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:18.534 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:18.534 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:18.559 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:18.560 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:19.323 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~70s 2026-02-25T08:42:20.561 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:20.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:20.636+0000 7f2f9a50b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:20.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:20.640+0000 7f2f9a50b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:20.645 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:20.645 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:20.670 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:20.670 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:20.737 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:20.810 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:20.812 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:20.813 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:22.671 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:22.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:22.747+0000 7ff6e78cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:22.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:22.750+0000 7ff6e78cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:22.755 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:22.755 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:22.783 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:22.783 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:24.725 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~76s 2026-02-25T08:42:24.784 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:24.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:24.858+0000 7f0f36c35640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:24.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:24.862+0000 7f0f36c35640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:24.867 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:24.867 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:24.891 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:24.892 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:25.815 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:25.886 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:25.889 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:25.890 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:26.892 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:26.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:26.968+0000 7fae9dff5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:26.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:26.972+0000 7fae9dff5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:26.976 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:26.976 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:27.003 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:27.004 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:29.005 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:29.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:29.080+0000 7fc6752c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:29.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:29.084+0000 7fc6752c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:29.088 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:29.089 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:29.116 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:29.117 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:30.129 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~81s 2026-02-25T08:42:30.891 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:30.962 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:30.964 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:30.965 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:31.118 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:31.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:31.191+0000 7f8798a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:31.195+0000 7f8793577640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:31.199 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:31.200 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:31.226 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:31.226 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:33.228 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:33.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:33.303+0000 7f8406dfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:33.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:33.307+0000 7f8406dfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:33.311 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:33.311 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:33.338 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:33.338 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:35.339 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:35.416 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:35.414+0000 7f37640cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:35.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:35.418+0000 7f37640cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:35.422 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:35.422 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:35.449 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:35.450 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:35.532 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~86s 2026-02-25T08:42:35.966 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:36.036 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:36.039 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:36.039 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:37.451 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:37.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:37.527+0000 7f611ba5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:37.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:37.530+0000 7f611ba5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:37.535 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:37.535 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:37.562 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:37.563 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:39.563 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:39.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:39.638+0000 7f9ea781a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:39.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:39.641+0000 7f9ea781a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:39.646 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:39.647 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:39.672 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:39.673 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:40.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~92s 2026-02-25T08:42:41.040 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:41.112 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:41.115 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:41.116 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:41.674 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:41.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:41.749+0000 7f6333ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:41.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:41.753+0000 7f6333ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:41.757 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:41.757 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:41.785 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:41.786 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:43.787 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:43.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:43.862+0000 7faf19011640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:43.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:43.865+0000 7faf19011640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:43.870 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:43.870 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:43.895 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:43.896 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:45.897 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:45.974 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:45.971+0000 7f40e9174640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:45.976 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:45.974+0000 7f40e9174640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:45.979 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:45.980 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:46.003 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:46.004 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:46.116 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:46.186 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:46.188 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:46.189 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:46.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~97s 2026-02-25T08:42:48.005 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:48.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:48.078+0000 7f533b70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:48.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:48.081+0000 7f533b70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:48.087 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:48.087 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:48.112 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:48.112 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:50.113 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:50.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:50.188+0000 7f14b859c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:50.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:50.192+0000 7f14b859c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:50.198 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:50.198 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:50.223 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:50.223 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:51.190 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:51.262 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:51.265 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:51.265 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:51.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~103s 2026-02-25T08:42:52.224 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:52.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:52.298+0000 7f0cc1ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:52.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:52.301+0000 7f0cc1ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:52.307 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:52.307 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:52.334 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:52.334 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:54.336 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:54.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:54.408+0000 7fe58e6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:54.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:54.412+0000 7fe58e6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:54.417 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:54.417 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:54.443 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:54.444 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:56.266 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:56.340 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:56.343 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:56.343 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:56.445 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:56.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:56.519+0000 7f58b7011640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:56.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:56.523+0000 7f58b7011640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:56.528 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:56.528 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:56.553 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:56.554 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:57.145 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~108s 2026-02-25T08:42:58.555 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:58.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:58.630+0000 7f8a53fb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:58.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:42:58.633+0000 7f8a53fb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:58.638 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:42:58.638 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:58.665 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:58.665 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:00.666 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:00.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:00.743+0000 7faab879c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:00.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:00.746+0000 7faab879c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:00.751 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:00.751 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:00.781 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:00.781 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:01.344 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:01.414 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:01.417 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:01.418 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:02.549 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~113s 2026-02-25T08:43:02.782 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:02.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:02.856+0000 7f6883c1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:02.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:02.860+0000 7f6883c1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:02.864 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:02.864 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:02.890 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:02.891 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:04.891 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:04.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:04.968+0000 7f8bff05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:04.972 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:04.971+0000 7f8bff05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:04.976 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:04.976 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:05.008 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:05.008 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:06.419 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:06.491 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:06.493 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:06.494 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:07.010 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:07.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:07.086+0000 7f7094211640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:07.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:07.089+0000 7f7094211640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:07.093 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:07.094 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:07.121 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:07.122 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:07.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~119s 2026-02-25T08:43:09.123 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:09.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:09.197+0000 7f67bd6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:09.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:09.200+0000 7f67bd6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:09.205 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:09.205 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:09.232 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:09.232 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:11.233 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:11.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:11.307+0000 7f838c81a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:11.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:11.311+0000 7f838c81a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:11.315 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:11.315 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:11.341 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:11.342 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:11.495 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:11.565 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:11.568 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:11.569 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:13.343 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:13.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~124s 2026-02-25T08:43:13.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:13.417+0000 7f79c625a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:13.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:13.420+0000 7f79c625a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:13.425 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:13.426 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:13.451 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:13.452 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:15.453 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:15.527+0000 7fb68b2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:15.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:15.530+0000 7fb68b2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:15.535 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:15.535 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:15.562 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:15.563 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:16.570 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:16.642 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:16.645 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:16.646 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:17.564 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:17.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:17.640+0000 7f6e40ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:17.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:17.643+0000 7f6e40ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:17.648 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:17.648 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:17.671 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:17.672 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:18.759 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~130s 2026-02-25T08:43:19.673 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:19.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:19.750+0000 7f27ad3a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:19.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:19.754+0000 7f27ad3a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:19.758 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:19.758 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:19.790 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:19.790 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:21.647 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:21.719 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:21.722 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:21.723 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:21.791 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:21.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:21.865+0000 7f919c564640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:21.869 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:21.869+0000 7f919c564640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:21.873 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:21.873 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:21.903 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:21.904 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:23.905 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:23.983 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:23.981+0000 7fdbff7b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:23.986 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:23.985+0000 7fdbff7b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:23.989 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:23.989 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:24.014 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:24.015 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:24.162 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~135s 2026-02-25T08:43:26.016 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:26.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:26.092+0000 7f4ff203d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:26.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:26.095+0000 7f4ff203d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:26.099 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:26.100 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:26.126 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:26.126 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:26.723 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:26.795 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:26.798 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:26.798 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:28.128 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:28.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:28.200+0000 7f0985e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:28.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:28.203+0000 7f0985e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:28.208 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:28.208 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:28.236 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:28.236 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:29.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~140s 2026-02-25T08:43:30.237 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:30.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:30.311+0000 7fc68f57e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:30.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:30.314+0000 7fc68f57e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:30.319 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:30.320 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:30.346 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:30.346 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:31.799 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:31.870 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:31.872 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:31.873 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:32.347 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:32.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:32.423+0000 7fa687a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:32.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:32.426+0000 7fa687a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:32.432 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:32.432 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:32.459 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:32.459 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:34.461 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:34.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:34.536+0000 7fd26130b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:34.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:34.539+0000 7fd26130b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:34.545 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:34.545 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:34.570 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:34.570 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:34.968 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~146s 2026-02-25T08:43:36.571 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:36.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:36.643+0000 7f5c852cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:36.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:36.646+0000 7f5c852cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:36.651 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:36.652 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:36.677 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:36.678 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:36.874 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:36.947 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:36.949 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:36.950 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:38.679 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:38.754+0000 7f77ded0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:38.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:38.758+0000 7f77ded0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:38.764 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:38.764 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:38.788 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:38.789 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:40.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~151s 2026-02-25T08:43:40.790 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:40.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:40.865+0000 7fedbddfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:40.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:40.868+0000 7fedbddfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:40.874 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:40.874 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:40.901 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:40.902 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:41.951 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:42.022 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:42.025 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:42.026 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:42.903 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:42.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:42.978+0000 7fe022ffd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:42.983 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:42.982+0000 7fe022ffd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:42.986 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:42.986 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:43.011 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:43.011 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:45.012 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:45.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:45.088+0000 7ff2f9d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:45.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:45.092+0000 7ff2f9d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:45.096 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:45.096 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:45.123 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:45.124 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:45.775 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~157s 2026-02-25T08:43:47.027 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:47.097 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:47.100 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:47.101 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:47.124 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:47.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:47.197+0000 7f1b0b85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:47.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:47.201+0000 7f1b0b85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:47.205 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:47.205 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:47.230 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:47.231 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:49.232 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:49.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:49.308+0000 7f0177b9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:49.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:49.312+0000 7f0177b9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:49.316 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:49.317 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:49.342 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:49.343 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:51.178 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~162s 2026-02-25T08:43:51.344 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:51.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:51.419+0000 7f9581acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:51.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:51.423+0000 7f9581acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:51.427 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:51.427 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:51.454 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:51.454 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:52.102 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:52.174 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:52.177 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:52.178 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:53.455 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:53.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:53.531+0000 7f521a90b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:53.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:53.535+0000 7f521a90b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:53.539 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:53.539 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:53.568 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:53.569 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:55.570 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:55.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:55.645+0000 7fac22ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:55.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:55.649+0000 7fac22ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:55.653 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:55.653 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:55.679 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:55.680 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:56.581 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~168s 2026-02-25T08:43:57.179 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:57.253 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:57.256 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:57.256 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:57.680 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:57.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:57.755+0000 7f686fecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:57.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:57.759+0000 7f686fecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:57.763 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:57.763 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:57.791 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:57.791 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:59.793 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:59.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:59.869+0000 7f53f950b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:59.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:43:59.873+0000 7f53f950b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:59.877 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:43:59.878 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:59.903 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:59.904 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:01.905 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:01.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:01.980+0000 7f1b2e90b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:01.984 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~173s 2026-02-25T08:44:01.985 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:01.983+0000 7f1b2e90b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:01.988 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:01.988 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:02.015 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:02.015 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:02.257 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:02.328 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:02.330 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:02.331 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:04.016 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:04.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:04.091+0000 7f3f3939c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:04.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:04.094+0000 7f3f3939c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:04.099 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:04.099 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:04.129 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:04.130 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:06.131 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:06.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:06.208+0000 7f301e1e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:06.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:06.211+0000 7f301e1e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:06.216 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:06.216 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:06.242 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:06.243 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:07.332 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:07.387 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~178s 2026-02-25T08:44:07.404 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:07.407 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:07.408 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:08.244 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:08.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:08.317+0000 7f711e85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:08.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:08.321+0000 7f711e85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:08.326 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:08.326 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:08.358 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:08.359 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:10.360 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:10.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:10.434+0000 7f541645a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:10.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:10.438+0000 7f541645a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:10.443 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:10.444 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:10.469 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:10.470 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:12.409 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:12.471 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:12.479 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:12.482 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:12.482 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:12.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:12.544+0000 7ff3cd246640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:12.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:12.548+0000 7ff3cd246640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:12.553 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:12.553 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:12.580 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:12.581 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:12.790 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~184s 2026-02-25T08:44:14.581 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:14.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:14.656+0000 7fe36021a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:14.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:14.660+0000 7fe36021a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:14.665 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:14.666 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:14.692 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:14.693 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:16.694 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:16.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:16.769+0000 7f5c4e5c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:16.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:16.772+0000 7f5c4e5c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:16.777 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:16.778 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:16.806 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:16.806 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:17.483 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:17.554 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:17.557 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:17.558 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:18.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~189s 2026-02-25T08:44:18.808 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:18.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:18.883+0000 7f6beb5fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:18.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:18.887+0000 7f6beb5fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:18.892 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:18.892 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:18.919 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:18.920 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:20.921 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:21.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:20.997+0000 7f24cadb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:21.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:21.000+0000 7f24cadb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:21.006 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:21.006 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:21.032 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:21.033 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:22.559 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:22.631 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:22.634 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:22.635 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:23.034 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:23.108+0000 7f867b81a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:23.111+0000 7f867b81a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:23.116 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:23.116 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:23.141 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:23.141 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:23.597 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~195s 2026-02-25T08:44:25.143 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:25.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:25.220+0000 7f747bb0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:25.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:25.223+0000 7f747bb0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:25.228 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:25.228 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:25.253 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:25.254 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:27.255 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:27.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:27.332+0000 7fb7feb93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:27.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:27.336+0000 7fb7feb93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:27.340 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:27.341 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:27.365 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:27.365 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:27.636 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:27.708 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:27.710 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:27.711 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:29.001 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~200s 2026-02-25T08:44:29.367 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:29.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:29.442+0000 7f982901a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:29.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:29.447+0000 7f982901a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:29.452 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:29.452 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:29.479 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:29.479 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:31.480 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:31.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:31.557+0000 7faa5d5ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:31.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:31.561+0000 7faa5d5ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:31.565 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:31.566 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:31.593 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:31.594 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:32.712 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:32.784 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:32.787 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:32.788 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:33.595 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:33.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:33.670+0000 7fe23cbdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:33.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:33.674+0000 7fe23cbdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:33.678 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:33.679 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:33.711 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:33.712 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:34.405 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~205s 2026-02-25T08:44:35.713 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:35.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:35.788+0000 7f0ea499c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:35.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:35.791+0000 7f0ea499c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:35.796 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:35.796 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:35.822 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:35.823 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:37.789 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:37.824 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:37.861 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:37.864 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:37.864 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:37.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:37.904+0000 7fd673d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:37.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:37.907+0000 7fd673d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:37.912 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:37.912 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:37.940 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:37.940 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:39.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~211s 2026-02-25T08:44:39.941 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:40.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:40.017+0000 7f7362e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:40.022 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:40.021+0000 7f7362e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:40.026 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:40.026 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:40.052 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:40.053 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:42.053 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:42.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:42.129+0000 7fb703ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:42.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:42.133+0000 7fb703ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:42.137 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:42.137 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:42.164 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:42.164 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:42.866 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:42.939 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:42.941 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:42.942 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:44.165 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:44.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:44.241+0000 7f2208da4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:44.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:44.244+0000 7f2208da4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:44.249 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:44.249 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:44.275 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:44.276 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:45.211 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~216s 2026-02-25T08:44:46.276 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:46.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:46.351+0000 7f2c4ee1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:46.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:46.355+0000 7f2c4ee1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:46.359 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:46.359 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:46.383 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:46.384 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:47.943 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:48.015 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:48.018 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:48.019 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:48.385 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:48.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:48.460+0000 7f4417e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:48.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:48.463+0000 7f4417e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:48.469 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:48.469 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:48.495 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:48.496 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:50.497 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:50.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:50.571+0000 7fe5a2cc2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:50.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:50.575+0000 7fe5a2cc2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:50.580 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:50.581 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:50.605 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:50.605 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:50.615 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~222s 2026-02-25T08:44:52.607 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:52.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:52.682+0000 7f4167846640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:52.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:52.686+0000 7f4167846640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:52.691 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:52.691 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:52.718 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:52.719 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:53.020 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:53.092 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:53.095 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:53.095 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:54.720 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:54.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:54.795+0000 7f39338c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:54.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:54.798+0000 7f39338c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:54.804 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:54.804 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:54.831 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:54.832 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:56.017 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~227s 2026-02-25T08:44:56.834 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:56.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:56.907+0000 7f48f4db8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:56.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:56.911+0000 7f48f4db8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:56.916 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:56.917 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:56.953 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:56.954 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:58.096 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:58.168 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:58.170 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:58.171 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:58.955 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:59.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:59.030+0000 7f78ff246640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:59.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:44:59.034+0000 7f78ff246640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:59.039 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:44:59.040 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:59.067 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:59.068 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:01.069 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:01.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:01.144+0000 7ff925793640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:01.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:01.148+0000 7ff925793640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:01.153 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:01.153 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:01.177 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:01.178 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:01.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~232s 2026-02-25T08:45:03.172 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:03.178 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:03.244 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:03.248 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:03.248 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:03.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:03.289+0000 7fbd496cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:03.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:03.292+0000 7fbd496cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:03.297 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:03.298 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:03.323 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:03.323 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:05.324 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:05.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:05.399+0000 7f7490fd3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:05.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:05.402+0000 7f7490fd3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:05.407 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:05.407 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:05.432 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:05.432 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:06.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~238s 2026-02-25T08:45:07.433 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:07.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:07.507+0000 7fa4dda5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:07.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:07.511+0000 7fa4dda5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:07.516 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:07.516 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:07.541 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:07.541 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:08.249 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:08.319 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:08.321 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:08.322 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:09.542 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:09.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:09.616+0000 7fa1a3774640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:09.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:09.620+0000 7fa1a3774640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:09.624 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:09.625 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:09.651 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:09.652 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:11.653 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:11.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:11.729+0000 7f7578dfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:11.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:11.733+0000 7f7578dfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:11.738 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:11.738 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:11.763 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:11.764 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:12.228 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~243s 2026-02-25T08:45:13.323 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:13.397 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:13.400 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:13.400 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:13.764 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:13.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:13.840+0000 7f35bc76f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:13.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:13.843+0000 7f35bc76f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:13.847 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:13.848 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:13.876 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:13.876 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:15.877 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:15.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:15.954+0000 7f657e4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:15.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:15.958+0000 7f657e4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:15.962 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:15.962 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:15.988 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:15.989 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:17.631 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~249s 2026-02-25T08:45:17.990 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:18.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:18.067+0000 7fedff23d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:18.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:18.070+0000 7fedff23d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:18.075 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:18.075 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:18.102 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:18.102 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:18.401 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:18.472 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:18.475 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:18.476 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:20.103 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:20.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:20.178+0000 7f74cb4c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:20.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:20.182+0000 7f74cb4c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:20.187 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:20.187 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:20.212 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:20.213 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:22.214 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:22.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:22.290+0000 7f8a81564640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:22.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:22.293+0000 7f8a81564640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:22.298 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:22.298 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:22.323 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:22.323 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:23.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~254s 2026-02-25T08:45:23.476 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:23.549 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:23.552 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:23.552 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:24.324 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:24.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:24.399+0000 7efdd01e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:24.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:24.402+0000 7efdd01e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:24.407 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:24.407 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:24.433 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:24.434 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:26.434 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:26.511+0000 7f5e2441a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:26.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:26.515+0000 7f5e2441a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:26.521 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:26.521 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:26.546 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:26.546 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:28.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~259s 2026-02-25T08:45:28.548 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:28.553 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:28.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:28.621+0000 7f59e481a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:28.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:28.625+0000 7f59e481a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:28.631 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:28.631 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:28.657 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:28.657 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:28.660 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:28.663 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:28.663 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:30.659 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:30.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:30.731+0000 7fe2e681a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:30.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:30.735+0000 7fe2e681a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:30.740 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:30.740 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:30.765 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:30.765 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:32.766 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:32.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:32.843+0000 7fe21e85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:32.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:32.846+0000 7fe21e85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:32.852 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:32.852 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:32.879 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:32.879 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:33.665 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:33.736 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:33.739 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:33.740 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:33.843 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~265s 2026-02-25T08:45:34.880 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:34.954+0000 7f56a9fc1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:34.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:34.958+0000 7f56a9fc1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:34.964 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:34.964 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:34.988 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:34.989 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:36.989 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:37.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:37.065+0000 7fe95d0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:37.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:37.069+0000 7fe95d0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:37.074 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:37.074 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:37.100 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:37.100 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:38.741 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:38.812 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:38.815 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:38.815 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:39.101 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:39.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:39.177+0000 7f23883dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:39.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:39.180+0000 7f23883dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:39.186 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:39.186 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:39.213 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:39.214 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:39.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~270s 2026-02-25T08:45:41.214 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:41.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:41.290+0000 7f88de5c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:41.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:41.294+0000 7f88de5c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:41.299 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:41.299 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:41.325 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:41.326 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:43.327 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:43.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:43.402+0000 7f7c171dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:43.406 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:43.405+0000 7f7c171dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:43.410 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:43.410 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:43.437 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:43.438 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:43.816 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:43.889 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:43.892 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:43.893 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:44.649 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~276s 2026-02-25T08:45:45.439 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:45.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:45.516+0000 7f80e0983640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:45.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:45.520+0000 7f80e0983640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:45.524 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:45.525 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:45.550 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:45.551 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:47.551 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:47.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:47.628+0000 7f3f5aa5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:47.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:47.631+0000 7f3f5aa5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:47.636 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:47.636 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:47.662 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:47.663 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:48.893 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:48.966 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:48.969 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:48.970 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:49.664 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:49.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:49.737+0000 7fb60fa46640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:49.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:49.741+0000 7fb60fa46640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:49.745 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:49.745 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:49.772 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:49.773 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:50.053 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~281s 2026-02-25T08:45:51.774 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:51.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:51.850+0000 7f78dab0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:51.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:51.854+0000 7f78dab0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:51.859 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:51.859 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:51.889 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:51.889 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:53.891 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:53.970 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:53.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:53.969+0000 7ff3f8b5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:53.974 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:53.973+0000 7ff3f8b5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:53.978 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:53.978 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:54.005 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:54.005 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:54.041 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:54.043 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:54.044 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:55.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~286s 2026-02-25T08:45:56.006 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:56.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:56.082+0000 7f5fd1ddc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:56.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:56.086+0000 7f5fd1ddc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:56.090 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:56.090 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:56.115 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:56.116 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:58.117 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:58.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:58.193+0000 7f4bbad0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:58.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:45:58.197+0000 7f4bbad0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:58.202 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:45:58.202 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:58.227 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:58.228 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:59.045 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:59.117 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:59.120 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:59.120 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:00.229 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:00.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:00.304+0000 7fb1117e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:00.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:00.308+0000 7fb1117e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:00.313 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:46:00.313 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:00.337 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:00.338 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:00.859 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~292s 2026-02-25T08:46:02.339 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:02.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:02.415+0000 7f913f411640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:02.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:02.418+0000 7f913f411640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:02.422 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:46:02.422 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:02.447 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:02.448 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:04.121 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:04.192 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:04.195 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:04.195 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:04.449 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:04.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:04.524+0000 7fb832c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:04.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:04.528+0000 7fb832c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:04.532 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:46:04.533 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:04.560 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:04.561 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:06.262 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~297s 2026-02-25T08:46:06.562 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:06.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:06.637+0000 7f76748cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:06.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:06.641+0000 7f76748cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:06.646 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:46:06.646 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:06.671 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:06.671 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:08.672 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:08.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:08.747+0000 7f2c941b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:08.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:08.750+0000 7f2c941b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:08.755 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:46:08.756 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:08.784 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:08.785 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:09.196 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:09.269 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:09.272 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:09.272 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:10.786 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:10.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:10.861+0000 7f087ab6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:10.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:46:10.864+0000 7f087ab6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:10.870 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-25T08:46:10.870 INFO:teuthology.orchestra.run.trial006.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:10.897 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:10.897 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:11.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~303s 2026-02-25T08:46:11.665 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2026-02-25T08:46:12.068 ERROR:tasks.ceph.osd.0:No such daemon running 2026-02-25T08:46:12.069 INFO:tasks.ceph.osd.2:Sent signal 15 2026-02-25T08:46:12.070 INFO:tasks.ceph.osd.3:Sent signal 15 2026-02-25T08:46:12.070 INFO:tasks.ceph.mon.a:Sent signal 15 2026-02-25T08:46:12.070 INFO:tasks.ceph.mgr.x:Sent signal 15 2026-02-25T08:46:12.070 INFO:tasks.daemonwatchdog.daemon_watchdog:Killing thrasher OSDThrasher 2026-02-25T08:46:12.071 INFO:tasks.ceph.mon.a.trial006.stderr:2026-02-25T08:46:12.069+0000 7f59987fc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 40602) UID: 0 2026-02-25T08:46:12.071 INFO:tasks.ceph.mon.a.trial006.stderr:2026-02-25T08:46:12.069+0000 7f59987fc640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-25T08:46:12.080 INFO:tasks.ceph.mgr.x.trial158.stderr:2026-02-25T08:46:12.076+0000 7f6dd9ff8640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 39742) UID: 0 2026-02-25T08:46:12.081 INFO:tasks.ceph.mgr.x.trial158.stderr:2026-02-25T08:46:12.076+0000 7f6dd9ff8640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2026-02-25T08:46:14.273 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:14.344 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:14.347 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:14.348 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:19.349 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:19.421 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:19.424 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:19.424 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:24.425 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:24.497 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:24.500 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:24.501 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:29.502 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:29.574 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:29.577 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:29.578 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:34.579 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:34.654 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:34.657 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:34.658 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:39.659 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:39.730 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:39.733 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:39.734 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:44.735 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:44.807 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:44.810 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:44.810 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:49.811 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:49.881 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:49.884 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:49.885 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:54.886 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:54.958 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:54.961 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:54.961 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:59.962 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:00.035 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:00.038 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:00.038 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:47:05.039 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:05.110 INFO:teuthology.orchestra.run.trial006.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:05.113 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:05.117 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 192, in wrapper return func(self) ^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1513, in _do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 562, in revive_osd self.ceph_manager.revive_osd( File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 3236, in revive_osd self.wait_run_admin_socket('osd', osd, File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2085, in wait_run_admin_socket raise Exception('timed out waiting for admin_socket ' Exception: timed out waiting for admin_socket to appear after osd.1 restart 2026-02-25T08:47:05.117 ERROR:tasks.thrashosds.thrasher:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1384, in do_thrash self._do_thrash() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 192, in wrapper return func(self) ^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1513, in _do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 562, in revive_osd self.ceph_manager.revive_osd( File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 3236, in revive_osd self.wait_run_admin_socket('osd', osd, File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2085, in wait_run_admin_socket raise Exception('timed out waiting for admin_socket ' Exception: timed out waiting for admin_socket to appear after osd.1 restart 2026-02-25T08:47:05.119 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2026-02-25T08:47:05.119 INFO:tasks.daemonwatchdog.daemon_watchdog:Killing remote process ceph-test-rados-ceph 2026-02-25T08:47:05.119 DEBUG:tasks.rados:Stopping ceph-test-rados-ceph due to exception The WatchDog barked due to Daemon ceph.osd.1 has failed 2026-02-25T08:47:05.119 INFO:tasks.rados:Stopping instance 0 2026-02-25T08:47:05.127 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 352, in thread raise watched_process.exception tasks.daemonwatchdog.BarkError: The WatchDog barked due to Daemon ceph.osd.1 has failed 2026-02-25T08:47:05.129 ERROR:teuthology.run_tasks:Manager failed: rados Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 372, in task running.get() File "src/gevent/greenlet.py", line 831, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 352, in thread raise watched_process.exception ^^^^^^^^^^^^^^^^^ tasks.daemonwatchdog.BarkError: The WatchDog barked due to Daemon ceph.osd.1 has failed 2026-02-25T08:47:05.131 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2026-02-25T08:47:05.147 INFO:tasks.thrashosds:joining thrashosds 2026-02-25T08:47:05.147 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2026-02-25T08:47:05.147 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-25T08:47:05.147 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T08:47:05.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:47:05.240+0000 7fea1e3c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:49:05.198 DEBUG:teuthology.orchestra.run:got remote process result: 124 2026-02-25T08:49:05.198 ERROR:teuthology.run_tasks:Manager failed: thrashosds Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 214, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 372, in task running.get() File "src/gevent/greenlet.py", line 831, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 352, in thread raise watched_process.exception ^^^^^^^^^^^^^^^^^ tasks.daemonwatchdog.BarkError: The WatchDog barked due to Daemon ceph.osd.1 has failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 218, in task cluster_manager.wait_for_all_osds_up() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2970, in wait_for_all_osds_up while not self.are_all_osds_up(): ^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2960, in are_all_osds_up x = self.get_osd_dump() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2670, in get_osd_dump return self.get_osd_dump_json()['osds'] ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2662, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial006 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2026-02-25T08:49:05.203 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-25T08:49:05.227 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-25T08:49:05.227 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:49:05.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-25T08:49:05.301+0000 7f962f19c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:51:05.259 DEBUG:teuthology.orchestra.run:got remote process result: 124 2026-02-25T08:51:05.260 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2000, in task yield File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 214, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 372, in task running.get() File "src/gevent/greenlet.py", line 831, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 352, in thread raise watched_process.exception ^^^^^^^^^^^^^^^^^ tasks.daemonwatchdog.BarkError: The WatchDog barked due to Daemon ceph.osd.1 has failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial006 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2026-02-25T08:51:05.265 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-25T08:51:05.265 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-25T08:51:05.265 ERROR:tasks.ceph.osd.0:tried to stop a non-running daemon 2026-02-25T08:51:05.266 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-25T08:51:05.266 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:51:05.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:51:05.266 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2000, in task yield File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 214, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 372, in task running.get() File "src/gevent/greenlet.py", line 831, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 352, in thread raise watched_process.exception ^^^^^^^^^^^^^^^^^ tasks.daemonwatchdog.BarkError: The WatchDog barked due to Daemon ceph.osd.1 has failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 1525, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial006 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial006 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log' 2026-02-25T08:51:05.271 INFO:tasks.ceph.osd.1:Stopped 2026-02-25T08:51:05.272 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-25T08:51:05.272 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:51:05.335 INFO:tasks.ceph.osd.2:Stopped 2026-02-25T08:51:05.336 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-25T08:51:05.336 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:51:05.400 INFO:tasks.ceph.osd.3:Stopped 2026-02-25T08:51:05.400 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-25T08:51:05.401 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-25T08:51:05.401 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:51:05.401 INFO:tasks.ceph.mgr.x:Stopped 2026-02-25T08:51:05.401 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-25T08:51:05.401 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-25T08:51:05.402 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:51:05.402 INFO:tasks.ceph.mon.a:Stopped 2026-02-25T08:51:05.402 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-25T08:51:05.403 DEBUG:teuthology.orchestra.run.trial006:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v 'but it is still running' | grep -E -v 'objects unfound and apparently lost' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSDMAP_FLAGS\)' | grep -E -v '\(OSD_' | grep -E -v '\(PG_' | grep -E -v '\(POOL_' | grep -E -v '\(CACHE_POOL_' | grep -E -v '\(SMALLER_PGP_NUM\)' | grep -E -v '\(OBJECT_' | grep -E -v SLOW_OPS | grep -E -v '\(REQUEST_SLOW\)' | grep -E -v '\(TOO_FEW_PGS\)' | grep -E -v 'slow request' | grep -E -v 'timeout on replica' | grep -E -v 'late reservation from' | grep -E -v MON_DOWN | grep -E -v OSDMAP_FLAGS | grep -E -v OSD_DOWN | grep -E -v PG_DEGRADED | grep -E -v PG_AVAILABILITY | grep -E -v POOL_APP_NOT_ENABLED | grep -E -v 'mons down' | grep -E -v 'mon down' | grep -E -v 'out of quorum' | grep -E -v noscrub | grep -E -v nodeep-scrub | grep -E -v 'Degraded data redundancy' | grep -E -v 'is down' | grep -E -v 'osds down' | grep -E -v 'pg .*? is .*?degraded.*?, acting' | grep -E -v 'pg .*? is stuck' | grep -E -v 'pg degraded' | grep -E -v PG_BACKFILL_FULL | grep -E -v 'Low space hindering backfill .*? backfill_toofull' | grep -E -v OSD_ROOT_DOWN | grep -E -v 'pgs degraded' | grep -E -v 'pgs undersized' | grep -E -v 'is active.*backfill_toofull.*' | head -n 1 2026-02-25T08:51:05.433 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial006.front.sepia.ceph.com 2026-02-25T08:51:05.434 DEBUG:teuthology.orchestra.run.trial006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-25T08:51:05.500 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial006.front.sepia.ceph.com 2026-02-25T08:51:05.501 DEBUG:teuthology.orchestra.run.trial006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-25T08:51:05.568 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial158.front.sepia.ceph.com 2026-02-25T08:51:05.569 DEBUG:teuthology.orchestra.run.trial158:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-25T08:51:05.601 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial158.front.sepia.ceph.com 2026-02-25T08:51:05.601 DEBUG:teuthology.orchestra.run.trial158:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-25T08:51:05.668 INFO:tasks.ceph:Archiving mon data... 2026-02-25T08:51:05.669 DEBUG:teuthology.misc:Transferring archived files from trial006:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69032/data/mon.a.tgz 2026-02-25T08:51:05.670 DEBUG:teuthology.orchestra.run.trial006:> mktemp 2026-02-25T08:51:05.687 INFO:teuthology.orchestra.run.trial006.stdout:/tmp/tmp.kh0Xfuj9zy 2026-02-25T08:51:05.687 DEBUG:teuthology.orchestra.run.trial006:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.kh0Xfuj9zy 2026-02-25T08:51:05.784 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod 0666 /tmp/tmp.kh0Xfuj9zy 2026-02-25T08:51:05.864 DEBUG:teuthology.orchestra.remote:trial006:/tmp/tmp.kh0Xfuj9zy is 284KB 2026-02-25T08:51:05.933 DEBUG:teuthology.orchestra.run.trial006:> rm -fr /tmp/tmp.kh0Xfuj9zy 2026-02-25T08:51:05.948 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-25T08:51:05.948 DEBUG:teuthology.orchestra.run.trial006:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-25T08:51:05.991 DEBUG:teuthology.orchestra.run.trial158:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-25T08:51:06.070 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-25T08:51:06.086 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-25T08:51:06.086 INFO:tasks.ceph:Archiving crash dumps... 2026-02-25T08:51:06.089 DEBUG:teuthology.misc:Transferring archived files from trial006:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69032/remote/trial006/crash 2026-02-25T08:51:06.090 DEBUG:teuthology.orchestra.run.trial006:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-25T08:51:06.135 DEBUG:teuthology.misc:Transferring archived files from trial158:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69032/remote/trial158/crash 2026-02-25T08:51:06.136 DEBUG:teuthology.orchestra.run.trial158:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-25T08:51:06.161 INFO:tasks.ceph:Compressing logs... 2026-02-25T08:51:06.161 DEBUG:teuthology.orchestra.run.trial006:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T08:51:06.179 DEBUG:teuthology.orchestra.run.trial158:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T08:51:06.199 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49496.log 2026-02-25T08:51:06.200 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45681.log 2026-02-25T08:51:06.200 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48765.log 2026-02-25T08:51:06.200 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49496.log.gz 2026-02-25T08:51:06.200 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45681.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-25T08:51:06.200 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45681.log.gz 2026-02-25T08:51:06.200 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47037.log 2026-02-25T08:51:06.201 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48765.log.gz 2026-02-25T08:51:06.201 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44857.log 2026-02-25T08:51:06.201 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45575.log 2026-02-25T08:51:06.201 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47037.log.gz 2026-02-25T08:51:06.201 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51224.log 2026-02-25T08:51:06.201 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44857.log.gz 2026-02-25T08:51:06.202 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47622.log 2026-02-25T08:51:06.202 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45575.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45575.log.gz 2026-02-25T08:51:06.203 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51224.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-25T08:51:06.203 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51224.log.gz 2026-02-25T08:51:06.203 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44804.log 2026-02-25T08:51:06.203 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47622.log.gz 2026-02-25T08:51:06.203 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42404.log 2026-02-25T08:51:06.203 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.44804.log: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.44804.log.gz 2026-02-25T08:51:06.203 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50905.log 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51490.log 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42404.log.gz 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50905.log: 50.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.50905.log.gz 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48074.log 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43633.log 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51490.log.gz 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48074.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48074.log.gz 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48313.log 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42300.log 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43633.log.gz 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48313.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.48313.log.gz 2026-02-25T08:51:06.204 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44990.log 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45442.log 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42300.log.gz 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44990.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44990.log.gz 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48978.log 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45229.log 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45442.log.gz 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48978.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48978.log.gz 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42861.log 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50719.log 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45229.log.gz 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41143.log 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42861.log.gz/var/log/ceph/ceph-client.admin.50719.log: 2026-02-25T08:51:06.206 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.50719.log.gz 2026-02-25T08:51:06.207 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45362.log 2026-02-25T08:51:06.207 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41143.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.41143.log.gz 2026-02-25T08:51:06.207 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50081.log 2026-02-25T08:51:06.207 INFO:teuthology.orchestra.run.trial006.stderr:gzip/var/log/ceph/ceph-client.admin.45362.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43766.log 2026-02-25T08:51:06.207 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45362.log.gz 2026-02-25T08:51:06.207 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50081.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50081.log.gz 2026-02-25T08:51:06.207 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44591.log 2026-02-25T08:51:06.207 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47808.log 2026-02-25T08:51:06.208 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43766.log.gz 2026-02-25T08:51:06.208 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44591.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44591.log.gz 2026-02-25T08:51:06.208 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47542.log 2026-02-25T08:51:06.208 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50187.log 2026-02-25T08:51:06.208 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47808.log.gz 2026-02-25T08:51:06.208 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47542.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47542.log.gz 2026-02-25T08:51:06.208 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49031.log 2026-02-25T08:51:06.208 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50533.log 2026-02-25T08:51:06.208 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.50187.log.gz 2026-02-25T08:51:06.208 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49031.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.49031.log.gz 2026-02-25T08:51:06.208 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46585.log 2026-02-25T08:51:06.209 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46399.log 2026-02-25T08:51:06.209 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50533.log.gz 2026-02-25T08:51:06.209 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46585.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46585.log.gz 2026-02-25T08:51:06.209 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42000.log 2026-02-25T08:51:06.209 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50852.log 2026-02-25T08:51:06.209 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46399.log.gz 2026-02-25T08:51:06.210 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42000.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42000.log.gz 2026-02-25T08:51:06.210 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41265.log 2026-02-25T08:51:06.210 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50852.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50852.log.gz 2026-02-25T08:51:06.210 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44405.log 2026-02-25T08:51:06.210 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41608.log/var/log/ceph/ceph-client.admin.41265.log: 2026-02-25T08:51:06.210 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41265.log.gz 2026-02-25T08:51:06.210 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44405.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44405.log.gz 2026-02-25T08:51:06.210 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45894.log 2026-02-25T08:51:06.210 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43447.log 2026-02-25T08:51:06.210 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41608.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41608.log.gz 2026-02-25T08:51:06.211 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51038.log 2026-02-25T08:51:06.211 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45894.log.gz 2026-02-25T08:51:06.211 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43447.log: gzip -5 --verbose -- 50.4% /var/log/ceph/ceph-client.admin.46080.log 2026-02-25T08:51:06.211 INFO:teuthology.orchestra.run.trial006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43447.log.gz 2026-02-25T08:51:06.211 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44910.log 2026-02-25T08:51:06.211 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51038.log.gz 2026-02-25T08:51:06.212 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46080.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46080.log.gz 2026-02-25T08:51:06.212 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49762.log 2026-02-25T08:51:06.212 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50400.log 2026-02-25T08:51:06.212 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44910.log.gz 2026-02-25T08:51:06.212 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47223.log 2026-02-25T08:51:06.212 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49762.log.gz 2026-02-25T08:51:06.212 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45761.log 2026-02-25T08:51:06.212 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50400.log.gz 2026-02-25T08:51:06.212 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44352.log 2026-02-25T08:51:06.213 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47223.log.gz 2026-02-25T08:51:06.213 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42914.log 2026-02-25T08:51:06.213 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45761.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45761.log.gz 2026-02-25T08:51:06.213 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46266.log 2026-02-25T08:51:06.213 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44352.log.gz 2026-02-25T08:51:06.213 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45495.log 2026-02-25T08:51:06.213 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42914.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42914.log.gz 2026-02-25T08:51:06.213 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46266.log: gzip -5 50.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46266.log.gz 2026-02-25T08:51:06.214 INFO:teuthology.orchestra.run.trial006.stderr: /var/log/ceph/ceph-client.admin.50453.log 2026-02-25T08:51:06.214 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48260.log 2026-02-25T08:51:06.214 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45495.log.gz 2026-02-25T08:51:06.214 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50453.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50453.log.gz -5 2026-02-25T08:51:06.214 INFO:teuthology.orchestra.run.trial006.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48898.log 2026-02-25T08:51:06.214 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43314.log 2026-02-25T08:51:06.214 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.48260.log.gz 2026-02-25T08:51:06.214 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48898.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48898.log.gz 2026-02-25T08:51:06.214 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44138.log 2026-02-25T08:51:06.215 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47755.log 2026-02-25T08:51:06.215 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43314.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43314.log.gz 2026-02-25T08:51:06.215 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48659.log 2026-02-25T08:51:06.215 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44138.log.gz 2026-02-25T08:51:06.215 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48712.log 2026-02-25T08:51:06.215 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47755.log.gz 2026-02-25T08:51:06.216 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48659.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48659.log.gz 2026-02-25T08:51:06.216 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41314.log 2026-02-25T08:51:06.216 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48712.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48712.log.gz 2026-02-25T08:51:06.216 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41690.log 2026-02-25T08:51:06.216 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49443.log 2026-02-25T08:51:06.216 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41314.log.gz 2026-02-25T08:51:06.216 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41690.log: gzip -5 --verbose 49.6% -- /var/log/ceph/ceph-client.admin.43899.log -- replaced with /var/log/ceph/ceph-client.admin.41690.log.gz 2026-02-25T08:51:06.216 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-25T08:51:06.217 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46718.log 2026-02-25T08:51:06.217 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49443.log.gz 2026-02-25T08:51:06.217 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43899.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43899.log.gz 2026-02-25T08:51:06.217 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48579.log 2026-02-25T08:51:06.217 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46718.log: 50.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.46718.log.gz 2026-02-25T08:51:06.217 INFO:teuthology.orchestra.run.trial006.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51171.log 2026-02-25T08:51:06.218 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48579.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48579.log.gz 2026-02-25T08:51:06.218 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47941.log 2026-02-25T08:51:06.218 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51171.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51171.log.gz 2026-02-25T08:51:06.218 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44271.log 2026-02-25T08:51:06.218 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47941.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47941.log.gz 2026-02-25T08:51:06.218 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43952.log 2026-02-25T08:51:06.218 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51091.log 2026-02-25T08:51:06.218 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44271.log.gz 2026-02-25T08:51:06.218 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43952.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43952.log.gz 2026-02-25T08:51:06.219 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41046.log 2026-02-25T08:51:06.219 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41687.log 2026-02-25T08:51:06.219 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51091.log.gz 2026-02-25T08:51:06.219 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41046.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41046.log.gz 2026-02-25T08:51:06.219 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50958.log 2026-02-25T08:51:06.219 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45176.log 2026-02-25T08:51:06.219 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41687.log.gz 2026-02-25T08:51:06.219 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50958.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.50958.log.gz 2026-02-25T08:51:06.220 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51357.log 2026-02-25T08:51:06.220 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50772.log 2026-02-25T08:51:06.220 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45176.log.gz 2026-02-25T08:51:06.220 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50320.log 2026-02-25T08:51:06.220 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51357.log.gz 2026-02-25T08:51:06.220 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50772.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50772.log.gz 2026-02-25T08:51:06.220 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49177.log 2026-02-25T08:51:06.221 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47675.log 2026-02-25T08:51:06.221 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50320.log.gz 2026-02-25T08:51:06.221 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41902.log 2026-02-25T08:51:06.221 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49177.log.gz 2026-02-25T08:51:06.221 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49629.log 2026-02-25T08:51:06.221 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47675.log.gz 2026-02-25T08:51:06.221 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45628.log 2026-02-25T08:51:06.222 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41902.log.gz 2026-02-25T08:51:06.222 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49629.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49629.log.gz -5 2026-02-25T08:51:06.222 INFO:teuthology.orchestra.run.trial006.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43580.log 2026-02-25T08:51:06.222 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47356.log 2026-02-25T08:51:06.222 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45628.log.gz 2026-02-25T08:51:06.222 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43580.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43580.log.gz 2026-02-25T08:51:06.222 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43128.log 2026-02-25T08:51:06.223 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42049.log 2026-02-25T08:51:06.223 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47356.log.gz 2026-02-25T08:51:06.223 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43128.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43128.log.gz 2026-02-25T08:51:06.223 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47303.log 2026-02-25T08:51:06.223 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42299.log 2026-02-25T08:51:06.223 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42049.log.gz 2026-02-25T08:51:06.223 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47303.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47303.log.gz 2026-02-25T08:51:06.224 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46984.log 2026-02-25T08:51:06.224 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49097.log 2026-02-25T08:51:06.224 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42299.log.gz 2026-02-25T08:51:06.224 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46984.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46984.log.gz 2026-02-25T08:51:06.224 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46904.log 2026-02-25T08:51:06.224 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47489.log 2026-02-25T08:51:06.224 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49097.log.gz 2026-02-25T08:51:06.225 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46904.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46904.log.gz 2026-02-25T08:51:06.225 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49815.log 2026-02-25T08:51:06.225 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51840.log 2026-02-25T08:51:06.225 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47489.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47489.log.gz 2026-02-25T08:51:06.225 INFO:teuthology.orchestra.run.trial158.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-25T08:51:06.225 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49815.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49815.log.gz 2026-02-25T08:51:06.225 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43181.log 2026-02-25T08:51:06.225 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41492.log 2026-02-25T08:51:06.226 INFO:teuthology.orchestra.run.trial006.stderr: 7.2% -- replaced with /var/log/ceph/ceph-client.admin.51840.log.gz 2026-02-25T08:51:06.226 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43181.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43181.log.gz 2026-02-25T08:51:06.226 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42174.log 2026-02-25T08:51:06.226 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41492.log: /var/log/ceph/ceph-client.admin.42994.log 2026-02-25T08:51:06.226 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41492.log.gz 2026-02-25T08:51:06.226 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42174.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42174.log.gz 2026-02-25T08:51:06.226 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48393.log 2026-02-25T08:51:06.227 INFO:teuthology.orchestra.run.trial158.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-25T08:51:06.227 INFO:teuthology.orchestra.run.trial158.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39842.log 2026-02-25T08:51:06.227 INFO:teuthology.orchestra.run.trial158.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39889.log 2026-02-25T08:51:06.227 INFO:teuthology.orchestra.run.trial158.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39994.log 2026-02-25T08:51:06.227 INFO:teuthology.orchestra.run.trial158.stderr:/var/log/ceph/ceph-client.admin.39842.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39842.log.gz 2026-02-25T08:51:06.227 INFO:teuthology.orchestra.run.trial158.stderr:/var/log/ceph/ceph-client.admin.39889.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39889.log.gz 2026-02-25T08:51:06.227 INFO:teuthology.orchestra.run.trial158.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39943.log 2026-02-25T08:51:06.227 INFO:teuthology.orchestra.run.trial158.stderr:/var/log/ceph/ceph-client.admin.39994.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-25T08:51:06.227 INFO:teuthology.orchestra.run.trial158.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39994.log.gz 2026-02-25T08:51:06.227 INFO:teuthology.orchestra.run.trial158.stderr:/var/log/ceph/ceph-client.admin.39943.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39943.log.gz 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48446.log 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42994.log.gz 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48393.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48393.log.gz 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49948.log 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51878.log 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48446.log.gz 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49948.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49948.log.gz 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40972.log 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44032.log 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr: 8.0% -- replaced with /var/log/ceph/ceph-client.admin.51878.log.gz 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.40972.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40972.log.gz 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45043.log 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43500.log 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44032.log.gz 2026-02-25T08:51:06.228 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45043.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45043.log.gz 2026-02-25T08:51:06.229 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-25T08:51:06.229 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45309.log 2026-02-25T08:51:06.229 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43500.log.gz 2026-02-25T08:51:06.229 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44085.log 2026-02-25T08:51:06.229 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45309.log: 87.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-25T08:51:06.229 INFO:teuthology.orchestra.run.trial006.stderr: 49.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.45309.log.gz -5 2026-02-25T08:51:06.229 INFO:teuthology.orchestra.run.trial006.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48845.log 2026-02-25T08:51:06.229 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50001.log 2026-02-25T08:51:06.229 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44085.log.gz 2026-02-25T08:51:06.230 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48845.log: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.48845.log.gz 2026-02-25T08:51:06.230 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46452.log 2026-02-25T08:51:06.230 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2026-02-25T08:51:06.230 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50001.log.gz 2026-02-25T08:51:06.230 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46452.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46452.log.gz 2026-02-25T08:51:06.230 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47436.log 2026-02-25T08:51:06.230 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43367.log 2026-02-25T08:51:06.230 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2026-02-25T08:51:06.230 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47436.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47436.log.gz 2026-02-25T08:51:06.230 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41491.log 2026-02-25T08:51:06.231 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50134.log 2026-02-25T08:51:06.231 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43367.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43367.log.gz 2026-02-25T08:51:06.231 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41491.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41491.log.gz 2026-02-25T08:51:06.231 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48127.log 2026-02-25T08:51:06.231 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40946.log 2026-02-25T08:51:06.231 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.50134.log.gz 2026-02-25T08:51:06.231 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48127.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.48127.log.gz 2026-02-25T08:51:06.232 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41363.log 2026-02-25T08:51:06.232 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.40946.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40946.log.gz 2026-02-25T08:51:06.232 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-25T08:51:06.232 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41363.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41363.log.gz 2026-02-25T08:51:06.232 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45123.log 2026-02-25T08:51:06.232 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46346.log 2026-02-25T08:51:06.232 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42098.log 2026-02-25T08:51:06.233 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45123.log.gz 2026-02-25T08:51:06.233 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47170.log 2026-02-25T08:51:06.233 INFO:teuthology.orchestra.run.trial006.stderr: 50.0%/var/log/ceph/ceph-client.admin.42098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41493.log 2026-02-25T08:51:06.233 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42098.log.gz 2026-02-25T08:51:06.233 INFO:teuthology.orchestra.run.trial006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46346.log.gz 2026-02-25T08:51:06.233 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46133.log 2026-02-25T08:51:06.234 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47170.log.gz 2026-02-25T08:51:06.234 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48526.log 2026-02-25T08:51:06.234 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41493.log.gz 2026-02-25T08:51:06.234 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41216.log 2026-02-25T08:51:06.234 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46133.log.gz 2026-02-25T08:51:06.234 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42550.log 2026-02-25T08:51:06.234 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48526.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48526.log.gz 2026-02-25T08:51:06.235 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47888.log 2026-02-25T08:51:06.235 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41216.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41216.log.gz 2026-02-25T08:51:06.235 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51304.log 2026-02-25T08:51:06.235 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42550.log.gz 2026-02-25T08:51:06.235 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.47888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42606.log 2026-02-25T08:51:06.235 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47888.log.gz 2026-02-25T08:51:06.235 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51304.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.39543.log 2026-02-25T08:51:06.235 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51304.log.gz 2026-02-25T08:51:06.236 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48207.log 2026-02-25T08:51:06.236 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42606.log.gz 2026-02-25T08:51:06.236 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph.tmp-client.admin.39543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46532.log 2026-02-25T08:51:06.236 INFO:teuthology.orchestra.run.trial006.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.39543.log.gz 2026-02-25T08:51:06.236 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46000.log 2026-02-25T08:51:06.236 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48207.log.gz 2026-02-25T08:51:06.236 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49230.log 2026-02-25T08:51:06.236 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46532.log.gz 2026-02-25T08:51:06.237 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46000.log: 49.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.46000.log.gz 2026-02-25T08:51:06.237 INFO:teuthology.orchestra.run.trial006.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44218.log 2026-02-25T08:51:06.237 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41951.log 2026-02-25T08:51:06.237 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49230.log.gz 2026-02-25T08:51:06.237 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41804.log 2026-02-25T08:51:06.237 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44218.log.gz 2026-02-25T08:51:06.237 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46771.log 2026-02-25T08:51:06.238 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41951.log.gz 2026-02-25T08:51:06.238 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43261.log 2026-02-25T08:51:06.238 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41804.log.gz 2026-02-25T08:51:06.238 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45814.log 2026-02-25T08:51:06.238 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46771.log.gz 2026-02-25T08:51:06.238 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44671.log 2026-02-25T08:51:06.238 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.43261.log.gz 2026-02-25T08:51:06.239 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41853.log 2026-02-25T08:51:06.239 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45814.log.gz 2026-02-25T08:51:06.239 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49310.log 2026-02-25T08:51:06.239 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44671.log.gz 2026-02-25T08:51:06.239 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49868.log 2026-02-25T08:51:06.239 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41853.log.gz 2026-02-25T08:51:06.239 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44458.log 2026-02-25T08:51:06.239 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49310.log.gz 2026-02-25T08:51:06.240 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46851.log 2026-02-25T08:51:06.240 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49868.log.gz 2026-02-25T08:51:06.240 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41490.log 2026-02-25T08:51:06.240 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44458.log.gz 2026-02-25T08:51:06.240 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47994.log 2026-02-25T08:51:06.240 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46851.log.gz 2026-02-25T08:51:06.241 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41490.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41490.log.gz 2026-02-25T08:51:06.241 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41688.log 2026-02-25T08:51:06.241 INFO:teuthology.orchestra.run.trial006.stderr:gzip/var/log/ceph/ceph-client.admin.47994.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42453.log 2026-02-25T08:51:06.241 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47994.log.gz 2026-02-25T08:51:06.241 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46665.log 2026-02-25T08:51:06.241 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41688.log.gz 2026-02-25T08:51:06.242 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42507.log 2026-02-25T08:51:06.242 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42453.log.gz 2026-02-25T08:51:06.242 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44724.log 2026-02-25T08:51:06.242 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46665.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46665.log.gz 2026-02-25T08:51:06.242 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.0.42507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40738.log 2026-02-25T08:51:06.242 INFO:teuthology.orchestra.run.trial006.stderr: 50.8% -- replaced with /var/log/ceph/ceph-client.0.42507.log.gz 2026-02-25T08:51:06.242 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43686.log 2026-02-25T08:51:06.242 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44724.log.gz 2026-02-25T08:51:06.242 INFO:teuthology.orchestra.run.trial006.stderr:gzip/var/log/ceph/ceph-client.admin.40738.log: -5 --verbose -- /var/log/ceph/ceph.log 2026-02-25T08:51:06.242 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40738.log.gz 2026-02-25T08:51:06.243 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43047.log 2026-02-25T08:51:06.243 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43686.log.gz 2026-02-25T08:51:06.243 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50586.log 2026-02-25T08:51:06.243 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42301.log 2026-02-25T08:51:06.243 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43047.log.gz 2026-02-25T08:51:06.243 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45947.log 2026-02-25T08:51:06.243 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50586.log.gz 2026-02-25T08:51:06.244 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.42301.log: 91.1% -- replaced with /var/log/ceph/ceph.log.gz 49.6% 2026-02-25T08:51:06.244 INFO:teuthology.orchestra.run.trial006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42301.log.gz 2026-02-25T08:51:06.244 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46213.log 2026-02-25T08:51:06.244 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40691.log 2026-02-25T08:51:06.244 INFO:teuthology.orchestra.run.trial006.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.45947.log.gz 2026-02-25T08:51:06.244 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51410.log 2026-02-25T08:51:06.244 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46213.log.gz 2026-02-25T08:51:06.244 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5/var/log/ceph/ceph-client.admin.40691.log: --verbose -- /var/log/ceph/ceph-client.admin.44538.log 2026-02-25T08:51:06.245 INFO:teuthology.orchestra.run.trial006.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40691.log.gz 2026-02-25T08:51:06.245 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43819.log 2026-02-25T08:51:06.245 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51410.log.gz 2026-02-25T08:51:06.245 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.44538.log: gzip -5 --verbose 50.0% -- /var/log/ceph/ceph-client.admin.49549.log -- replaced with /var/log/ceph/ceph-client.admin.44538.log.gz 2026-02-25T08:51:06.245 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-25T08:51:06.245 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.43819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49363.log 2026-02-25T08:51:06.245 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43819.log.gz 2026-02-25T08:51:06.245 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50267.log 2026-02-25T08:51:06.246 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49549.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.49549.log.gz 2026-02-25T08:51:06.246 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49682.log 2026-02-25T08:51:06.246 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49363.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49363.log.gz 2026-02-25T08:51:06.246 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50639.log 2026-02-25T08:51:06.246 INFO:teuthology.orchestra.run.trial006.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50267.log.gz 2026-02-25T08:51:06.246 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40605.log 2026-02-25T08:51:06.246 INFO:teuthology.orchestra.run.trial006.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49682.log.gz 2026-02-25T08:51:06.246 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50639.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50639.log.gz 2026-02-25T08:51:06.246 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41116.log 2026-02-25T08:51:06.246 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.40605.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40605.log.gz 2026-02-25T08:51:06.246 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.41116.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41116.log.gz 2026-02-25T08:51:06.255 INFO:teuthology.orchestra.run.trial158.stderr:/var/log/ceph/ceph-mgr.x.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-25T08:51:06.304 INFO:teuthology.orchestra.run.trial006.stderr: 89.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-25T08:51:06.545 INFO:teuthology.orchestra.run.trial006.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-25T08:51:06.618 INFO:teuthology.orchestra.run.trial158.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-25T08:51:06.621 INFO:teuthology.orchestra.run.trial158.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-25T08:51:06.622 INFO:teuthology.orchestra.run.trial158.stderr: 2026-02-25T08:51:06.622 INFO:teuthology.orchestra.run.trial158.stderr:real 0m0.405s 2026-02-25T08:51:06.623 INFO:teuthology.orchestra.run.trial158.stderr:user 0m0.772s 2026-02-25T08:51:06.623 INFO:teuthology.orchestra.run.trial158.stderr:sys 0m0.064s 2026-02-25T08:51:06.733 INFO:teuthology.orchestra.run.trial006.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-25T08:51:06.736 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-25T08:51:06.736 INFO:teuthology.orchestra.run.trial006.stderr:real 0m0.544s 2026-02-25T08:51:06.736 INFO:teuthology.orchestra.run.trial006.stderr:user 0m0.943s 2026-02-25T08:51:06.736 INFO:teuthology.orchestra.run.trial006.stderr:sys 0m0.133s 2026-02-25T08:51:06.736 INFO:tasks.ceph:Archiving logs... 2026-02-25T08:51:06.737 DEBUG:teuthology.misc:Transferring archived files from trial006:/var/log/ceph to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69032/remote/trial006/log 2026-02-25T08:51:06.740 DEBUG:teuthology.orchestra.run.trial006:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T08:51:07.066 DEBUG:teuthology.misc:Transferring archived files from trial158:/var/log/ceph to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69032/remote/trial158/log 2026-02-25T08:51:07.067 DEBUG:teuthology.orchestra.run.trial158:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T08:51:07.260 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2000, in task yield File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 214, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 372, in task running.get() File "src/gevent/greenlet.py", line 831, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 352, in thread raise watched_process.exception ^^^^^^^^^^^^^^^^^ tasks.daemonwatchdog.BarkError: The WatchDog barked due to Daemon ceph.osd.1 has failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial006 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2026-02-25T08:51:07.261 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-25T08:51:07.272 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/task/install/__init__.py", line 644, in task yield File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2000, in task yield File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 214, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 372, in task running.get() File "src/gevent/greenlet.py", line 831, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 352, in thread raise watched_process.exception ^^^^^^^^^^^^^^^^^ tasks.daemonwatchdog.BarkError: The WatchDog barked due to Daemon ceph.osd.1 has failed 2026-02-25T08:51:07.391 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-25T08:51:07.402 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/task/install/__init__.py", line 644, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 372, in task running.get() File "src/gevent/greenlet.py", line 831, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/rados.py", line 352, in thread raise watched_process.exception ^^^^^^^^^^^^^^^^^ tasks.daemonwatchdog.BarkError: The WatchDog barked due to Daemon ceph.osd.1 has failed 2026-02-25T08:51:07.403 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-25T08:51:07.403 DEBUG:teuthology.orchestra.run.trial006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T08:51:07.405 DEBUG:teuthology.orchestra.run.trial158:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T08:51:07.431 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-25T08:51:07.443 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-25T08:51:07.443 DEBUG:teuthology.orchestra.run.trial006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T08:51:07.473 DEBUG:teuthology.orchestra.run.trial158:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T08:51:07.486 INFO:teuthology.orchestra.run.trial006.stderr:bash: line 1: ntpq: command not found 2026-02-25T08:51:07.487 INFO:teuthology.orchestra.run.trial158.stderr:bash: line 1: ntpq: command not found 2026-02-25T08:51:07.517 INFO:teuthology.orchestra.run.trial006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T08:51:07.517 INFO:teuthology.orchestra.run.trial006.stdout:=============================================================================== 2026-02-25T08:51:07.517 INFO:teuthology.orchestra.run.trial006.stdout:^- ntpool0.603.newcontinuum> 2 8 377 7 -763us[ -763us] +/- 43ms 2026-02-25T08:51:07.517 INFO:teuthology.orchestra.run.trial006.stdout:^* time.cloudflare.com 3 8 377 146 -37us[ -186us] +/- 6650us 2026-02-25T08:51:07.517 INFO:teuthology.orchestra.run.trial006.stdout:^- 96-19-94-82.cpe.sparklig> 1 9 377 8 +388us[ +388us] +/- 32ms 2026-02-25T08:51:07.517 INFO:teuthology.orchestra.run.trial006.stdout:^- 170-187-142-180.ip.linod> 2 8 377 14 -11ms[ -11ms] +/- 79ms 2026-02-25T08:51:07.520 INFO:teuthology.orchestra.run.trial158.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T08:51:07.520 INFO:teuthology.orchestra.run.trial158.stdout:=============================================================================== 2026-02-25T08:51:07.520 INFO:teuthology.orchestra.run.trial158.stdout:^* time.lmtlabs.com 2 8 377 203 +1614us[+1640us] +/- 15ms 2026-02-25T08:51:07.520 INFO:teuthology.orchestra.run.trial158.stdout:^- 73.185.182.209 2 9 377 13 +262us[ +262us] +/- 44ms 2026-02-25T08:51:07.520 INFO:teuthology.orchestra.run.trial158.stdout:^- usmoi.meezy.link 2 7 377 66 -2406us[-2406us] +/- 23ms 2026-02-25T08:51:07.520 INFO:teuthology.orchestra.run.trial158.stdout:^+ 173.249.203.227 2 7 377 9 -2179us[-2179us] +/- 31ms 2026-02-25T08:51:07.521 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-25T08:51:07.537 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-25T08:51:07.537 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-25T08:51:07.553 DEBUG:teuthology.orchestra.run.trial006:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T08:51:07.615 DEBUG:teuthology.orchestra.run.trial158:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T08:51:07.671 DEBUG:teuthology.orchestra.run.trial006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T08:51:07.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:51:07.699 DEBUG:teuthology.orchestra.run.trial158:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T08:51:07.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:51:07.725 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-25T08:51:07.736 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-25T08:51:07.757 INFO:teuthology.task.internal:Duration was 2847.152966 seconds 2026-02-25T08:51:07.758 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-25T08:51:07.769 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-25T08:51:07.769 DEBUG:teuthology.orchestra.run.trial006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T08:51:07.771 DEBUG:teuthology.orchestra.run.trial158:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T08:51:07.806 INFO:teuthology.orchestra.run.trial158.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T08:51:07.807 INFO:teuthology.orchestra.run.trial006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T08:51:08.134 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-25T08:51:08.134 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial006.front.sepia.ceph.com 2026-02-25T08:51:08.135 DEBUG:teuthology.orchestra.run.trial006:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T08:51:08.155 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial158.front.sepia.ceph.com 2026-02-25T08:51:08.156 DEBUG:teuthology.orchestra.run.trial158:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T08:51:08.192 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-25T08:51:08.192 DEBUG:teuthology.orchestra.run.trial006:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T08:51:08.198 DEBUG:teuthology.orchestra.run.trial158:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T08:51:08.292 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-25T08:51:08.292 DEBUG:teuthology.orchestra.run.trial006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T08:51:08.295 DEBUG:teuthology.orchestra.run.trial158:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T08:51:08.316 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:51:08.316 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:51:08.317 INFO:teuthology.orchestra.run.trial006.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T08:51:08.317 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T08:51:08.317 INFO:teuthology.orchestra.run.trial006.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T08:51:08.318 INFO:teuthology.orchestra.run.trial158.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:51:08.319 INFO:teuthology.orchestra.run.trial158.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:51:08.319 INFO:teuthology.orchestra.run.trial158.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T08:51:08.319 INFO:teuthology.orchestra.run.trial158.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T08:51:08.319 INFO:teuthology.orchestra.run.trial158.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T08:51:08.327 INFO:teuthology.orchestra.run.trial006.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 83.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T08:51:08.328 INFO:teuthology.orchestra.run.trial158.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.3% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T08:51:08.330 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-25T08:51:08.348 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-25T08:51:08.348 DEBUG:teuthology.orchestra.run.trial006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T08:51:08.394 DEBUG:teuthology.orchestra.run.trial158:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T08:51:08.420 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-25T08:51:08.438 DEBUG:teuthology.orchestra.run.trial006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T08:51:08.441 DEBUG:teuthology.orchestra.run.trial158:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T08:51:08.464 INFO:teuthology.orchestra.run.trial006.stdout:kernel.core_pattern = core 2026-02-25T08:51:08.482 INFO:teuthology.orchestra.run.trial158.stdout:kernel.core_pattern = core 2026-02-25T08:51:08.498 DEBUG:teuthology.orchestra.run.trial006:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T08:51:08.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:51:08.538 DEBUG:teuthology.orchestra.run.trial158:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T08:51:08.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:51:08.552 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-25T08:51:08.570 INFO:teuthology.task.internal:Transferring archived files... 2026-02-25T08:51:08.572 DEBUG:teuthology.misc:Transferring archived files from trial006:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69032/remote/trial006 2026-02-25T08:51:08.572 DEBUG:teuthology.orchestra.run.trial006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T08:51:08.616 DEBUG:teuthology.misc:Transferring archived files from trial158:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69032/remote/trial158 2026-02-25T08:51:08.616 DEBUG:teuthology.orchestra.run.trial158:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T08:51:08.655 INFO:teuthology.task.internal:Removing archive directory... 2026-02-25T08:51:08.655 DEBUG:teuthology.orchestra.run.trial006:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T08:51:08.657 DEBUG:teuthology.orchestra.run.trial158:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T08:51:08.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-25T08:51:08.720 INFO:teuthology.task.internal:Not uploading archives. 2026-02-25T08:51:08.720 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-25T08:51:08.738 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-25T08:51:08.738 DEBUG:teuthology.orchestra.run.trial006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T08:51:08.741 DEBUG:teuthology.orchestra.run.trial158:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T08:51:08.756 INFO:teuthology.orchestra.run.trial006.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 08:51 /home/ubuntu/cephtest 2026-02-25T08:51:08.757 INFO:teuthology.orchestra.run.trial158.stdout: 917579 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 08:51 /home/ubuntu/cephtest 2026-02-25T08:51:08.759 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-25T08:51:08.778 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-25T08:51:08.798 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: BarkError: The WatchDog barked due to Daemon ceph.osd.1 has failed 2026-02-25T08:51:08.803 INFO:teuthology.run:Summary data: description: crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{default} thrashosds-health workloads/snaps-few-objects-localized} duration: 2847.1529655456543 failure_reason: The WatchDog barked due to Daemon ceph.osd.1 has failed flavor: debug owner: scheduled_matan@soko04.front.sepia.ceph.com status: fail success: false 2026-02-25T08:51:08.803 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T08:51:08.908 INFO:teuthology.run:FAIL